Can earnings conference calls tell more lies? A contrastive multimodal dialogue network for advanced financial statement fraud detection

Qi Lu, Wei Du*, Shaochen Yang, Wei Xu, J. Leon Zhao

*Corresponding author for this work

Research output: Journal Publications and ReviewsRGC 21 - Publication in refereed journalpeer-review

Abstract

Financial statement frauds by listed firms pose significant challenges to public investors and jeopardize the stability of financial markets. Previous studies have identified deceptive verbal and vocal cues from earnings conference calls as indicators of financial statement fraud. However, these studies only extracted managers' verbal and vocal cues separately over the entire call, neglecting the utterance-level fusion between verbal and vocal cues as well as the multi-turn interaction between analysts and managers. To fill this gap, we develop a novel end-to-end contrastive multimodal dialogue network (CMMD) that considers both verbal-vocal fusion and multi-role interactions to uncover hidden deceptive cues in earnings conference calls. The proposed model comprises two core modules, namely, the Multimodal Fusion Learning module and the Dialogue Interaction Learning module. Building on Vrij's verbal-nonverbal complementary mechanisms in deception detection, the designed Multimodal Fusion Learning employs contrastive learning to align verbal and vocal cues and a co-attention mechanism to learn cross-modal interaction. Inspired by the Interpersonal Deception Theory that emphasizes the dynamic interaction process between deceivers and targets, the Dialogue Interaction Learning utilizes a dialogue-aware co-attention mechanism to model multi-turn analyst-manager interaction and uses contrastive learning to improve dialogue representations. Our extensive empirical results show that CMMD achieves 8.64 % improvement in detecting fraudulent cases compared to the best baseline model. As such, our study advances the research frontier in fraud detection and contributes an innovative IT artifact in practice. © 2024 Published by Elsevier B.V.
Original languageEnglish
Article number114381
JournalDecision Support Systems
Volume189
Online published9 Dec 2024
DOIs
Publication statusPublished - Feb 2025

Bibliographical note

Research Unit(s) information for this publication is provided by the author(s) concerned.

Research Keywords

  • Financial statement fraud detection
  • Earnings conference calls
  • Q&A dialogues
  • Co-attention
  • Contrastive learning

Fingerprint

Dive into the research topics of 'Can earnings conference calls tell more lies? A contrastive multimodal dialogue network for advanced financial statement fraud detection'. Together they form a unique fingerprint.

Cite this