Skip to main navigation Skip to search Skip to main content

Decoding Multimodal Cues: Unveiling the Implicit Meaning behind Hateful Videos

  • Junyu Lu
  • , Deyi Ji*
  • , Liqun Liu
  • , Xiaokun Zhang
  • , Youlin Wu
  • , Roy Ka-Wei Lee
  • , Peng Shu
  • , Huan Yu
  • , Jie Jiang
  • , Bo Xu
  • , Liang Yang
  • , Hongfei Lin*
  • *Corresponding author for this work

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

2 Downloads (CityUHK Scholars)

Abstract

Hateful videos have become prevalent on online platforms, highlighting an urgent need for effective detection. However, existing studies primarily focus on binary classification and fail to provide contextual rationales that reveal the implicit meanings behind these judgments, significantly undermining model explainability. To fill this gap, we aim to achieve explainable hateful video detection, enabling models to provide contextual rationales that integrate relevant evidence and logical reasoning alongside decisions. This approach can comprehensively enhance the understanding of video content and the explainability of the decision-making process. We first introduce two datasets, Ex-HateMM and Ex-ImpliHateVid, for explainable hateful video detection. Each dataset provides fine-grained annotations of multimodal harmful elements, along with contextual rationales. We then propose an Information Augmentation and Reasoning Enhancement (IARE) framework designed for explainable detection. The framework employs an information augmentation phase that leverages the multimodal chain-of-thought to integrate harmful elements, thereby enriching rationale evidence. Additionally, IARE incorporates a reasoning enhancement phase, in which Direct Preference Optimization guides the model toward correct reasoning paths and away from incorrect ones, thereby improving the logical coherence of its justifications. We conduct extensive experiments on the two datasets, comparing multiple baselines with our proposed IARE framework. The results demonstrate that IARE achieves state-of-the-art performance while also generating accurate rationales. © 2026 Owner/Author.
Original languageEnglish
Title of host publicationSIGIR '26
Subtitle of host publicationProceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval
PublisherAssociation for Computing Machinery
Pages1255-1265
Number of pages11
ISBN (Print)979-8-4007-2599-9
DOIs
Publication statusPublished - Jul 2026
Event49th International SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2026) - Melbourne Convention and Exhibition Center, Melbourne | Naarm, Australia
Duration: 20 Jul 2026 → 24 Jul 2026
https://sigir2026.org/en-AU

Publication series

NameSIGIR - Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval

Conference

Conference49th International SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2026)
Abbreviated titleACM SIGIR 2026
PlaceAustralia
CityMelbourne | Naarm
Period20/07/26 → 24/07/26
Internet address

Bibliographical note

Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).

Funding

This research was supported in part by the National Natural Science Foundation of China (Nos. 625B2033, 62376051, and 62576073), the Ministry of Education Humanities and Social Science Project of China (No. 25YJCZH308), and the National Research Foundation, Prime Minister's Office, Singapore, together with the Ministry of Digital Development and Information, Singapore, under the Online Trust and Safety (OTS) Research Programme (Award No. S24T2TS007). The examples provided in this paper do not represent the views of the authors. Any conclusions do not necessarily reflect the views of the National Natural Science Foundation of China, the National Research Foundation, Prime Minister's Office, Singapore, or the Ministry of Digital Development and Information, Singapore.

Research Keywords

  • hateful video detection
  • large language model
  • multimodal understanding
  • reasoning enhancement

Publisher's Copyright Statement

  • This full text is made available under CC-BY-NC-ND 4.0. https://creativecommons.org/licenses/by-nc-nd/4.0/

Fingerprint

Dive into the research topics of 'Decoding Multimodal Cues: Unveiling the Implicit Meaning behind Hateful Videos'. Together they form a unique fingerprint.

Cite this