Nemesis : Combating Abusive Information in Encrypted Messaging with Private Reporting

Abusive messages spread rapidly within popular end-to-end encrypted messaging services (EEMSs) due to the inability to conduct content moderation on plaintext messages. This issue has turned EEMSs into breeding grounds for fake news, misinformation, and other viral abusive content. However, enabling the identification of abusive information without careful consideration can compromise the cherished confidentiality feature of EEMSs. To address this issue, we propose Nemesis, a privacy-friendly, user-reporting-based content moderation system for EEMSs. In Nemesis, users can privately report abusive messages, and the system traces the source if the report count exceeds a predefined threshold. Compared to existing systems, Nemesis offers three key advantages: 1) it supports flexible and adaptable thresholds to meet different content moderation needs, 2) it prevents dishonest reporters from disrupting the system, and 3) it better ensures the privacy of all users during the moderation process. Nemesis utilizes lightweight cryptographic techniques to achieve these goals, and evaluation results demonstrate its low overhead in securely identifying abusive messages and tracing their sources in EEMSs. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.