Towards Responsible Text Modeling

Abstract

This thesis focuses on advancing the field of responsible text modeling through three distinct research areas: interpretable machine translation, privacy-preserving machine translation, and toxicity detection for translation. The first two belong to the generation tasks in text modeling, focusing on the issues of interpretability and privacy protection within responsibility. The latter belongs to the text understanding task in text modeling, focusing on the issue of controlling harmful content within responsibility.

The first research contribution of this thesis is a novel framework for interpretable machine translation. A comprehensive training approach is proposed, which incorporates interpretable elements into standard neural machine translation models. The framework introduces efficient methods to provide insights into the translation process, enabling users to understand the decision-making mechanisms of the models.

The second research contribution is centered around privacy-preserving in the neural machine translation model. A benchmark including human-annotated datasets, evaluation metrics and a robust baseline framework is developed to bridge the gap in the privacy-preserving machine translation area. The proposed methods employ advanced cryptographic techniques, ensuring that user privacy is maintained without compromising the quality of translation outputs.

The final research contribution focuses on the generalization of the toxic text detection model, we explore the decision-making processes of existing toxic text detection models from lexical and semantic perspectives to make them transparent and investigate the potential of decoder-only large models in terms of generalization capabilities. This work contributes to creating a safer online environment by effectively detecting and mitigating harmful text.

Through these research contributions, this doctoral thesis expands the understanding and applications of responsible text modeling. The proposed framework for interpretable machine translation enhances transparency and understanding in language translation models and thus makes it more trustworthy. The privacy-preserving techniques address critical concerns in protecting user data during online translation services. Furthermore, the toxicity detection method contributes to fostering a healthier online communication environment.

Date of Award	6 Aug 2024
Original language	English
Awarding Institution	City University of Hong Kong
Supervisor	Linqi SONG (Supervisor)