TY - JOUR
T1 - Can earnings conference calls tell more lies? A contrastive multimodal dialogue network for advanced financial statement fraud detection
AU - Lu, Qi
AU - Du, Wei
AU - Yang, Shaochen
AU - Xu, Wei
AU - Zhao, J. Leon
N1 - Research Unit(s) information for this publication is provided by the author(s) concerned.
PY - 2025/2
Y1 - 2025/2
N2 - Financial statement frauds by listed firms pose significant challenges to public investors and jeopardize the stability of financial markets. Previous studies have identified deceptive verbal and vocal cues from earnings conference calls as indicators of financial statement fraud. However, these studies only extracted managers' verbal and vocal cues separately over the entire call, neglecting the utterance-level fusion between verbal and vocal cues as well as the multi-turn interaction between analysts and managers. To fill this gap, we develop a novel end-to-end contrastive multimodal dialogue network (CMMD) that considers both verbal-vocal fusion and multi-role interactions to uncover hidden deceptive cues in earnings conference calls. The proposed model comprises two core modules, namely, the Multimodal Fusion Learning module and the Dialogue Interaction Learning module. Building on Vrij's verbal-nonverbal complementary mechanisms in deception detection, the designed Multimodal Fusion Learning employs contrastive learning to align verbal and vocal cues and a co-attention mechanism to learn cross-modal interaction. Inspired by the Interpersonal Deception Theory that emphasizes the dynamic interaction process between deceivers and targets, the Dialogue Interaction Learning utilizes a dialogue-aware co-attention mechanism to model multi-turn analyst-manager interaction and uses contrastive learning to improve dialogue representations. Our extensive empirical results show that CMMD achieves 8.64 % improvement in detecting fraudulent cases compared to the best baseline model. As such, our study advances the research frontier in fraud detection and contributes an innovative IT artifact in practice. © 2024 Published by Elsevier B.V.
AB - Financial statement frauds by listed firms pose significant challenges to public investors and jeopardize the stability of financial markets. Previous studies have identified deceptive verbal and vocal cues from earnings conference calls as indicators of financial statement fraud. However, these studies only extracted managers' verbal and vocal cues separately over the entire call, neglecting the utterance-level fusion between verbal and vocal cues as well as the multi-turn interaction between analysts and managers. To fill this gap, we develop a novel end-to-end contrastive multimodal dialogue network (CMMD) that considers both verbal-vocal fusion and multi-role interactions to uncover hidden deceptive cues in earnings conference calls. The proposed model comprises two core modules, namely, the Multimodal Fusion Learning module and the Dialogue Interaction Learning module. Building on Vrij's verbal-nonverbal complementary mechanisms in deception detection, the designed Multimodal Fusion Learning employs contrastive learning to align verbal and vocal cues and a co-attention mechanism to learn cross-modal interaction. Inspired by the Interpersonal Deception Theory that emphasizes the dynamic interaction process between deceivers and targets, the Dialogue Interaction Learning utilizes a dialogue-aware co-attention mechanism to model multi-turn analyst-manager interaction and uses contrastive learning to improve dialogue representations. Our extensive empirical results show that CMMD achieves 8.64 % improvement in detecting fraudulent cases compared to the best baseline model. As such, our study advances the research frontier in fraud detection and contributes an innovative IT artifact in practice. © 2024 Published by Elsevier B.V.
KW - Financial statement fraud detection
KW - Earnings conference calls
KW - Q&A dialogues
KW - Co-attention
KW - Contrastive learning
UR - http://www.scopus.com/inward/record.url?scp=85212001248&partnerID=8YFLogxK
UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85212001248&origin=recordpage
U2 - 10.1016/j.dss.2024.114381
DO - 10.1016/j.dss.2024.114381
M3 - RGC 21 - Publication in refereed journal
SN - 0167-9236
VL - 189
JO - Decision Support Systems
JF - Decision Support Systems
M1 - 114381
ER -