Nemesis: Combating Abusive Information in Encrypted Messaging with Private Reporting

Rui Lian; Yulong Ming; Chengjun Cai; Yifeng Zheng; Cong Wang; Xiaohua Jia

doi:10.1007/978-3-031-70890-9_13

Nemesis: Combating Abusive Information in Encrypted Messaging with Private Reporting

Rui Lian
, Yulong Ming
, Chengjun Cai
, Yifeng Zheng
, Cong Wang^*
, Xiaohua Jia

^*Corresponding author for this work

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

2 Citations (Scopus)

Abstract

Abusive messages spread rapidly within popular end-to-end encrypted messaging services (EEMSs) due to the inability to conduct content moderation on plaintext messages. This issue has turned EEMSs into breeding grounds for fake news, misinformation, and other viral abusive content. However, enabling the identification of abusive information without careful consideration can compromise the cherished confidentiality feature of EEMSs. To address this issue, we propose Nemesis, a privacy-friendly, user-reporting-based content moderation system for EEMSs. In Nemesis, users can privately report abusive messages, and the system traces the source if the report count exceeds a predefined threshold. Compared to existing systems, Nemesis offers three key advantages: 1) it supports flexible and adaptable thresholds to meet different content moderation needs, 2) it prevents dishonest reporters from disrupting the system, and 3) it better ensures the privacy of all users during the moderation process. Nemesis utilizes lightweight cryptographic techniques to achieve these goals, and evaluation results demonstrate its low overhead in securely identifying abusive messages and tracing their sources in EEMSs. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.

Original language	English
Title of host publication	Computer Security – ESORICS 2024 - 29th European Symposium on Research in Computer Security, Proceedings
Editors	Joaquin Garcia-Alfaro, Rafał Kozik, Michał Choraś, Sokratis Katsikas
Publisher	Springer, Cham
Pages	247-267
Volume	Part II
ISBN (Electronic)	9783031708909
ISBN (Print)	9783031708893
DOIs	https://doi.org/10.1007/978-3-031-70890-9_13
Publication status	Published - 2024
Event	29th European Symposium on Research in Computer Security (ESORICS 2024) - Bydgoszcz University of Science and Technology, Bydgoszcz, Poland Duration: 16 Sept 2024 → 20 Sept 2024 https://esorics2024.org/

Publication series

Name	Lecture Notes in Computer Science
Volume	14983
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	29th European Symposium on Research in Computer Security (ESORICS 2024)
Place	Poland
City	Bydgoszcz
Period	16/09/24 → 20/09/24
Internet address	https://esorics2024.org/

Bibliographical note

Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).

Funding

We sincerely thank all anonymous reviewers for their useful comments and instructions. This work was supported in part by CityU of HK under Grants 9678146 and 9678126, by HK RGC under Grants CityU 11218521, 11218322, R6021-20F, R1012-21, RFS2122-1S04, C2004-21G, C1029-22G, and N_CityU139/21, by Guangdong Basic and Applied Basic Research Foundation under Grant 2023A1515010714 and 2023A1515140137, by Guangdong-Hong Kong Joint Laboratory for Data Security and Privacy Preserving under Grant 2023B1212120007, by the Shenzhen Science and Technology Program under Grants JCYJ20220531095416037 and JCYJ20230807094411024, by National Nature Science Foundation of China under Grant 62202398, and by InnoHK initiative, the Government of the HKSAR, and Laboratory for AI-Powered Financial Technologies.

Research Keywords

Content moderation
End-to-end encryption
Threshold aggregation

RGC Funding Information

RGC-funded

Access Output

10.1007/978-3-031-70890-9_13

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Lian, R., Ming, Y., Cai, C., Zheng, Y., Wang, C., & Jia, X. (2024). Nemesis: Combating Abusive Information in Encrypted Messaging with Private Reporting. In J. Garcia-Alfaro, R. Kozik, M. Choraś, & S. Katsikas (Eds.), Computer Security – ESORICS 2024 - 29th European Symposium on Research in Computer Security, Proceedings (Vol. Part II, pp. 247-267). (Lecture Notes in Computer Science; Vol. 14983). Springer, Cham. https://doi.org/10.1007/978-3-031-70890-9_13

Lian, Rui ; Ming, Yulong ; Cai, Chengjun et al. / Nemesis : Combating Abusive Information in Encrypted Messaging with Private Reporting. Computer Security – ESORICS 2024 - 29th European Symposium on Research in Computer Security, Proceedings. editor / Joaquin Garcia-Alfaro ; Rafał Kozik ; Michał Choraś ; Sokratis Katsikas. Vol. Part II Springer, Cham, 2024. pp. 247-267 (Lecture Notes in Computer Science).

@inproceedings{5e2f1e5f1d3f4c48ab1054d9ba744664,

title = "Nemesis: Combating Abusive Information in Encrypted Messaging with Private Reporting",

abstract = "Abusive messages spread rapidly within popular end-to-end encrypted messaging services (EEMSs) due to the inability to conduct content moderation on plaintext messages. This issue has turned EEMSs into breeding grounds for fake news, misinformation, and other viral abusive content. However, enabling the identification of abusive information without careful consideration can compromise the cherished confidentiality feature of EEMSs. To address this issue, we propose Nemesis, a privacy-friendly, user-reporting-based content moderation system for EEMSs. In Nemesis, users can privately report abusive messages, and the system traces the source if the report count exceeds a predefined threshold. Compared to existing systems, Nemesis offers three key advantages: 1) it supports flexible and adaptable thresholds to meet different content moderation needs, 2) it prevents dishonest reporters from disrupting the system, and 3) it better ensures the privacy of all users during the moderation process. Nemesis utilizes lightweight cryptographic techniques to achieve these goals, and evaluation results demonstrate its low overhead in securely identifying abusive messages and tracing their sources in EEMSs. {\textcopyright} The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.",

keywords = "Content moderation, End-to-end encryption, Threshold aggregation",

author = "Rui Lian and Yulong Ming and Chengjun Cai and Yifeng Zheng and Cong Wang and Xiaohua Jia",

note = "Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).; 29th European Symposium on Research in Computer Security (ESORICS 2024) ; Conference date: 16-09-2024 Through 20-09-2024",

year = "2024",

doi = "10.1007/978-3-031-70890-9\_13",

language = "English",

isbn = "9783031708893",

volume = "Part II",

series = "Lecture Notes in Computer Science",

publisher = "Springer, Cham",

pages = "247--267",

editor = "Joaquin Garcia-Alfaro and Rafa{\l} Kozik and Micha{\l} Chora{\'s} and Sokratis Katsikas",

booktitle = "Computer Security – ESORICS 2024 - 29th European Symposium on Research in Computer Security, Proceedings",

url = "https://esorics2024.org/",

}

Lian, R , Ming, Y , Cai, C , Zheng, Y , Wang, C & Jia, X 2024, Nemesis: Combating Abusive Information in Encrypted Messaging with Private Reporting. in J Garcia-Alfaro, R Kozik, M Choraś & S Katsikas (eds), Computer Security – ESORICS 2024 - 29th European Symposium on Research in Computer Security, Proceedings. vol. Part II, Lecture Notes in Computer Science, vol. 14983, Springer, Cham, pp. 247-267, 29th European Symposium on Research in Computer Security (ESORICS 2024), Bydgoszcz, Poland, 16/09/24. https://doi.org/10.1007/978-3-031-70890-9_13

Nemesis: Combating Abusive Information in Encrypted Messaging with Private Reporting. / Lian, Rui ; Ming, Yulong ; Cai, Chengjun et al.
Computer Security – ESORICS 2024 - 29th European Symposium on Research in Computer Security, Proceedings. ed. / Joaquin Garcia-Alfaro; Rafał Kozik; Michał Choraś; Sokratis Katsikas. Vol. Part II Springer, Cham, 2024. p. 247-267 (Lecture Notes in Computer Science; Vol. 14983).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Nemesis

T2 - 29th European Symposium on Research in Computer Security (ESORICS 2024)

AU - Lian, Rui

AU - Ming, Yulong

AU - Cai, Chengjun

AU - Zheng, Yifeng

AU - Wang, Cong

AU - Jia, Xiaohua

N1 - Full text of this publication does not contain sufficient affiliation information. With consent from the author(s) concerned, the Research Unit(s) information for this record is based on the existing academic department affiliation of the author(s).

PY - 2024

Y1 - 2024

N2 - Abusive messages spread rapidly within popular end-to-end encrypted messaging services (EEMSs) due to the inability to conduct content moderation on plaintext messages. This issue has turned EEMSs into breeding grounds for fake news, misinformation, and other viral abusive content. However, enabling the identification of abusive information without careful consideration can compromise the cherished confidentiality feature of EEMSs. To address this issue, we propose Nemesis, a privacy-friendly, user-reporting-based content moderation system for EEMSs. In Nemesis, users can privately report abusive messages, and the system traces the source if the report count exceeds a predefined threshold. Compared to existing systems, Nemesis offers three key advantages: 1) it supports flexible and adaptable thresholds to meet different content moderation needs, 2) it prevents dishonest reporters from disrupting the system, and 3) it better ensures the privacy of all users during the moderation process. Nemesis utilizes lightweight cryptographic techniques to achieve these goals, and evaluation results demonstrate its low overhead in securely identifying abusive messages and tracing their sources in EEMSs. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.

AB - Abusive messages spread rapidly within popular end-to-end encrypted messaging services (EEMSs) due to the inability to conduct content moderation on plaintext messages. This issue has turned EEMSs into breeding grounds for fake news, misinformation, and other viral abusive content. However, enabling the identification of abusive information without careful consideration can compromise the cherished confidentiality feature of EEMSs. To address this issue, we propose Nemesis, a privacy-friendly, user-reporting-based content moderation system for EEMSs. In Nemesis, users can privately report abusive messages, and the system traces the source if the report count exceeds a predefined threshold. Compared to existing systems, Nemesis offers three key advantages: 1) it supports flexible and adaptable thresholds to meet different content moderation needs, 2) it prevents dishonest reporters from disrupting the system, and 3) it better ensures the privacy of all users during the moderation process. Nemesis utilizes lightweight cryptographic techniques to achieve these goals, and evaluation results demonstrate its low overhead in securely identifying abusive messages and tracing their sources in EEMSs. © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.

KW - Content moderation

KW - End-to-end encryption

KW - Threshold aggregation

UR - https://www.scopus.com/pages/publications/85204575377

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85204575377&origin=recordpage

U2 - 10.1007/978-3-031-70890-9_13

DO - 10.1007/978-3-031-70890-9_13

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 9783031708893

VL - Part II

T3 - Lecture Notes in Computer Science

SP - 247

EP - 267

BT - Computer Security – ESORICS 2024 - 29th European Symposium on Research in Computer Security, Proceedings

A2 - Garcia-Alfaro, Joaquin

A2 - Kozik, Rafał

A2 - Choraś, Michał

A2 - Katsikas, Sokratis

PB - Springer, Cham

Y2 - 16 September 2024 through 20 September 2024

ER -

Lian R , Ming Y , Cai C , Zheng Y , Wang C , Jia X. Nemesis: Combating Abusive Information in Encrypted Messaging with Private Reporting. In Garcia-Alfaro J, Kozik R, Choraś M, Katsikas S, editors, Computer Security – ESORICS 2024 - 29th European Symposium on Research in Computer Security, Proceedings. Vol. Part II. Springer, Cham. 2024. p. 247-267. (Lecture Notes in Computer Science). Epub 2024 Sept 6. doi: 10.1007/978-3-031-70890-9_13

Nemesis: Combating Abusive Information in Encrypted Messaging with Private Reporting

Abstract

Publication series

Conference

Bibliographical note

Funding

Research Keywords

RGC Funding Information

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

CRF: Enabling Metadata-private and Accountable Networks at Scale

GRF: Towards Bridging the Last-mile Gap of Multi-party Collaborative Statistics

RIF: Secure and Incentivized Data Sharing with Enhanced Ownership for Decentralized Networks

Cite this

Nemesis: Combating Abusive Information in Encrypted Messaging with Private Reporting

Abstract

Publication series

Conference

Bibliographical note

Funding

Research Keywords

RGC Funding Information

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Projects

CRF: Enabling Metadata-private and Accountable Networks at Scale

GRF: Towards Bridging the Last-mile Gap of Multi-party Collaborative Statistics

RIF: Secure and Incentivized Data Sharing with Enhanced Ownership for Decentralized Networks

Cite this