Multimodal Lightweight Cross-Modal Attention Fusion for Efficient Hate Meme Detection

The proliferation of hateful memes on social media platforms presents significant challenges for content moderation, as these multimodal artifacts leverage the interplay between visual and textual elements to convey implicit offensive meanings. Existing detection approaches suffer from either inadequate cross-modal understanding or prohibitive computational costs unsuitable for real-world deployment. This paper proposes CMAFN-Lite (Cross-Modal Attention Fusion Network-Lite), a lightweight yet effective framework that achieves competitive hate speech detection performance while reducing model complexity by approximately 70%. Our approach employs bidirectional cross-modal attention mechanisms to capture semantic dependencies between image and text modalities, utilizing DistilBERT and MobileNetV3-Small as efficient feature extractors. Extensive experiments on the Hateful Memes dataset demonstrate that CMAFN-Lite achieves an F1-score of 0.599 and accuracy of 73.2% while maintaining only 28.6M parameters and accelerating training speed by 3.5× compared to heavyweight baselines. The proposed model achieves a favorable trade-off between performance and efficiency, making it suitable for deployment in resource-constrained environments. © 2025 Copyright held by the owner/author(s).