Uncertainty-Based Metamorphic Testing for Validating Plagiarism Detection Systems

Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45)32_Refereed conference paper (with host publication)peer-review

View graph of relations

Related Research Unit(s)

Detail(s)

Original languageEnglish
Title of host publicationTechnology in Education. Innovative Practices for the New Normal
Subtitle of host publication6th International Conference on Technology in Education, ICTE 2023, Hong Kong, China, December 19–21, 2023, Proceedings
EditorsSimon K. S. Cheung, Fu Lee Wang, Naraphorn Paoprasert, Peerayuth Charnsethikul, Kam Cheong Li, Kongkiti Phusavat
PublisherSpringer 
Pages299-314
ISBN (Electronic)978-981-99-8255-4
ISBN (Print)978-981-99-8254-7
Publication statusPublished - 2024

Publication series

NameCommunications in Computer and Information Science
Volume1974
ISSN (Print)1865-0929
ISSN (Electronic)1865-0937

Conference

Title6th International Conference on Technology in Education (ICTE 2023)
PlaceChina
CityHong Kong
Period19 - 21 December 2023

Abstract

Plagiarism is a severe issue in academia, and uncertainty in plagiarism detection systems might lead to inconsistent detections. Thus, evaluating the system is essential; however, it is also a test oracle problem as it is challenging to distinguish correct behaviour from potentially incorrect behaviour of the system. To alleviate this challenge, we develop a feasible approach by applying an uncertainty matrix to identify the uncertainty of the plagiarism detection systems and derive metamorphic relations of metamorphic testing from the identified uncertainty for validation. We experimented with three plagiarism detection systems in a classroom scenario where students were hypothesized to use tools to generate answers for assignments. These answers were fed into the systems for validation by comparing the systems’ similarity scores of the tool-generated answers. Results showed that the proposed approach can effectively validate plagiarism detection systems. Future studies can apply this approach to locate uncertainties to enhance systems’ robustness. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024

Research Area(s)

  • Validation, Metamorphic testing, Uncertainty, Natural language processing, Plagiarism detection

Citation Format(s)

Uncertainty-Based Metamorphic Testing for Validating Plagiarism Detection Systems. / Chan, Pak Yuen Patrick; Keung, Jacky; Yang, Zhen.
Technology in Education. Innovative Practices for the New Normal: 6th International Conference on Technology in Education, ICTE 2023, Hong Kong, China, December 19–21, 2023, Proceedings. ed. / Simon K. S. Cheung; Fu Lee Wang; Naraphorn Paoprasert; Peerayuth Charnsethikul; Kam Cheong Li; Kongkiti Phusavat. Springer , 2024. p. 299-314 (Communications in Computer and Information Science; Vol. 1974).

Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45)32_Refereed conference paper (with host publication)peer-review