Improving Readability Assessment with Ordinal Log-Loss

Research output: Chapters, Conference Papers, Creative and Literary WorksRGC 32 - Refereed conference paper (with host publication)peer-review

View graph of relations

Author(s)

Detail(s)

Original languageEnglish
Title of host publicationProceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024)
PublisherAssociation for Computational Linguistics
Pages343–350
Publication statusPublished - Jun 2024

Conference

Title19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024)
Location
PlaceMexico
CityMexico City
Period20 - 21 June 2024

Link(s)

Abstract

Automatic Readability Assessment (ARA) predicts the level of difficulty of a text, e.g. at Grade 1 to Grade 12. ARA is an ordinal classification task since the predicted levels follow an underlying order, from easy to difficult. However, most neural ARA models ignore the distance between the gold level and predicted level, treating all levels as independent labels. This paper investigates whether distance-sensitive loss functions can improve ARA performance. We evaluate a variety of loss functions on neural ARA models, and show that ordinal log-loss can produce statistically significant improvement over the standard cross-entropy loss in terms of adjacent accuracy in a majority of our datasets. © 2024 Association for Computational Linguistics

Research Area(s)

Citation Format(s)

Improving Readability Assessment with Ordinal Log-Loss. / Lim, Ho Hung; Lee, John S. Y.
Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024). Association for Computational Linguistics, 2024. p. 343–350.

Research output: Chapters, Conference Papers, Creative and Literary WorksRGC 32 - Refereed conference paper (with host publication)peer-review

Download Statistics

No data available