CompLex-ZH: A New Dataset for Lexical Complexity Prediction in Mandarin and Cantonese

Le Qiu; Shanyue Guo; Tak-sum Wong; Emmanuele Chersoni; John S. Y. Lee; Chu-Ren Huang

doi:10.18653/v1/2024.tsar-1.3

CompLex-ZH: A New Dataset for Lexical Complexity Prediction in Mandarin and Cantonese

Le Qiu
, Shanyue Guo
, Tak-sum Wong
, Emmanuele Chersoni^*
, John S. Y. Lee
, Chu-Ren Huang

^*Corresponding author for this work

Department of Linguistics and Translation

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

1 Citation (Scopus)

68 Downloads (CityUHK Scholars)

Abstract

The prediction of lexical complexity in context is assuming an increasing relevance in Natural Language Processing research, since identifying complex words is often the f irst step of text simplification pipelines. To the best of our knowledge, though, datasets annotated with complex words are available only for English and for a limited number of Western languages.

In our paper, we introduce CompLex-ZH, a dataset including words annotated with complexity scores in sentential contexts for Chinese. Our data include sentences in Mandarin and Cantonese, which were selected from a variety of sources and textual genres. We provide a first evaluation with baselines combining hand-crafted and language models-based features.

©2024 Association for Computational Linguistics

Original language	English
Title of host publication	Proceedings of the Third Workshop on Text Simplification, Accessibility and Readability (TSAR 2024)
Editors	Matthew Shardlow, Horacio Saggion, Fernando Alva-Manchego, Marcos Zampieri, Kai North, Sanja Štajner, Regina Stodden
Publisher	Association for Computational Linguistics
Pages	20-26
ISBN (Electronic)	979-8-89176-176-6
DOIs	https://doi.org/10.18653/v1/2024.tsar-1.3
Publication status	Published - Nov 2024
Event	29th Conference on Empirical Methods in Natural Language Processing (EMNLP 2024) - Hybrid, Miami, United States Duration: 12 Nov 2024 → 16 Nov 2024 https://2024.emnlp.org/

Conference

Conference	29th Conference on Empirical Methods in Natural Language Processing (EMNLP 2024)
Abbreviated title	EMNLP 2024
Place	United States
City	Miami
Period	12/11/24 → 16/11/24
Internet address	https://2024.emnlp.org/

Bibliographical note

Research Unit(s) information for this publication is provided by the author(s) concerned.

Funding

EC acknowledges the financial support from the start-up fund project “Building and Predicting Neurocognitive-Motivated Lexical-Semantic Norms for Mandarin Chinese”(1- BE8G), sponsored by the Faculty of Humanities of the Hong Kong Polytechnic University. JL acknowledges support from a Strategic Research Grant (7006037) at City University of Hong Kong.

Publisher's Copyright Statement

This full text is made available under CC-BY 4.0. https://creativecommons.org/licenses/by/4.0/

Access Output

10.18653/v1/2024.tsar-1.3

254731897.pdfFinal Published version, 432 KBLicence: CC BY

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Qiu, L., Guo, S., Wong, T., Chersoni, E., Lee, J. S. Y., & Huang, C.-R. (2024). CompLex-ZH: A New Dataset for Lexical Complexity Prediction in Mandarin and Cantonese. In M. Shardlow, H. Saggion, F. Alva-Manchego, M. Zampieri, K. North, S. Štajner, & R. Stodden (Eds.), Proceedings of the Third Workshop on Text Simplification, Accessibility and Readability (TSAR 2024) (pp. 20-26). Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.tsar-1.3

Qiu, Le ; Guo, Shanyue ; Wong, Tak-sum et al. / CompLex-ZH : A New Dataset for Lexical Complexity Prediction in Mandarin and Cantonese. Proceedings of the Third Workshop on Text Simplification, Accessibility and Readability (TSAR 2024). editor / Matthew Shardlow ; Horacio Saggion ; Fernando Alva-Manchego ; Marcos Zampieri ; Kai North ; Sanja Štajner ; Regina Stodden. Association for Computational Linguistics, 2024. pp. 20-26

@inproceedings{abebbc615d624f2bbd3c20221049b353,

title = "CompLex-ZH: A New Dataset for Lexical Complexity Prediction in Mandarin and Cantonese",

abstract = "The prediction of lexical complexity in context is assuming an increasing relevance in Natural Language Processing research, since identifying complex words is often the f irst step of text simplification pipelines. To the best of our knowledge, though, datasets annotated with complex words are available only for English and for a limited number of Western languages. In our paper, we introduce CompLex-ZH, a dataset including words annotated with complexity scores in sentential contexts for Chinese. Our data include sentences in Mandarin and Cantonese, which were selected from a variety of sources and textual genres. We provide a first evaluation with baselines combining hand-crafted and language models-based features. {\textcopyright}2024 Association for Computational Linguistics ",

author = "Le Qiu and Shanyue Guo and Tak-sum Wong and Emmanuele Chersoni and Lee, \{John S. Y.\} and Chu-Ren Huang",

note = "Research Unit(s) information for this publication is provided by the author(s) concerned.; 29th Conference on Empirical Methods in Natural Language Processing (EMNLP 2024), EMNLP 2024 ; Conference date: 12-11-2024 Through 16-11-2024",

year = "2024",

month = nov,

doi = "10.18653/v1/2024.tsar-1.3",

language = "English",

pages = "20--26",

editor = "Matthew Shardlow and Horacio Saggion and Fernando Alva-Manchego and Marcos Zampieri and Kai North and Sanja {\v S}tajner and Regina Stodden",

booktitle = "Proceedings of the Third Workshop on Text Simplification, Accessibility and Readability (TSAR 2024)",

publisher = "Association for Computational Linguistics",

address = "United States",

url = "https://2024.emnlp.org/",

}

Qiu, L, Guo, S, Wong, T, Chersoni, E, Lee, JSY & Huang, C-R 2024, CompLex-ZH: A New Dataset for Lexical Complexity Prediction in Mandarin and Cantonese. in M Shardlow, H Saggion, F Alva-Manchego, M Zampieri, K North, S Štajner & R Stodden (eds), Proceedings of the Third Workshop on Text Simplification, Accessibility and Readability (TSAR 2024). Association for Computational Linguistics, pp. 20-26, 29th Conference on Empirical Methods in Natural Language Processing (EMNLP 2024), Miami, Florida, United States, 12/11/24. https://doi.org/10.18653/v1/2024.tsar-1.3

CompLex-ZH: A New Dataset for Lexical Complexity Prediction in Mandarin and Cantonese. / Qiu, Le; Guo, Shanyue; Wong, Tak-sum et al.
Proceedings of the Third Workshop on Text Simplification, Accessibility and Readability (TSAR 2024). ed. / Matthew Shardlow; Horacio Saggion; Fernando Alva-Manchego; Marcos Zampieri; Kai North; Sanja Štajner; Regina Stodden. Association for Computational Linguistics, 2024. p. 20-26.

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - CompLex-ZH

T2 - 29th Conference on Empirical Methods in Natural Language Processing (EMNLP 2024)

AU - Qiu, Le

AU - Guo, Shanyue

AU - Wong, Tak-sum

AU - Chersoni, Emmanuele

AU - Lee, John S. Y.

AU - Huang, Chu-Ren

N1 - Research Unit(s) information for this publication is provided by the author(s) concerned.

PY - 2024/11

Y1 - 2024/11

N2 - The prediction of lexical complexity in context is assuming an increasing relevance in Natural Language Processing research, since identifying complex words is often the f irst step of text simplification pipelines. To the best of our knowledge, though, datasets annotated with complex words are available only for English and for a limited number of Western languages. In our paper, we introduce CompLex-ZH, a dataset including words annotated with complexity scores in sentential contexts for Chinese. Our data include sentences in Mandarin and Cantonese, which were selected from a variety of sources and textual genres. We provide a first evaluation with baselines combining hand-crafted and language models-based features. ©2024 Association for Computational Linguistics

AB - The prediction of lexical complexity in context is assuming an increasing relevance in Natural Language Processing research, since identifying complex words is often the f irst step of text simplification pipelines. To the best of our knowledge, though, datasets annotated with complex words are available only for English and for a limited number of Western languages. In our paper, we introduce CompLex-ZH, a dataset including words annotated with complexity scores in sentential contexts for Chinese. Our data include sentences in Mandarin and Cantonese, which were selected from a variety of sources and textual genres. We provide a first evaluation with baselines combining hand-crafted and language models-based features. ©2024 Association for Computational Linguistics

U2 - 10.18653/v1/2024.tsar-1.3

DO - 10.18653/v1/2024.tsar-1.3

M3 - RGC 32 - Refereed conference paper (with host publication)

SP - 20

EP - 26

BT - Proceedings of the Third Workshop on Text Simplification, Accessibility and Readability (TSAR 2024)

A2 - Shardlow, Matthew

A2 - Saggion, Horacio

A2 - Alva-Manchego, Fernando

A2 - Zampieri, Marcos

A2 - North, Kai

A2 - Štajner, Sanja

A2 - Stodden, Regina

PB - Association for Computational Linguistics

Y2 - 12 November 2024 through 16 November 2024

ER -

Qiu L, Guo S, Wong T, Chersoni E, Lee JSY, Huang CR. CompLex-ZH: A New Dataset for Lexical Complexity Prediction in Mandarin and Cantonese. In Shardlow M, Saggion H, Alva-Manchego F, Zampieri M, North K, Štajner S, Stodden R, editors, Proceedings of the Third Workshop on Text Simplification, Accessibility and Readability (TSAR 2024). Association for Computational Linguistics. 2024. p. 20-26 doi: 10.18653/v1/2024.tsar-1.3

CompLex-ZH: A New Dataset for Lexical Complexity Prediction in Mandarin and Cantonese

Abstract

Conference

Bibliographical note

Funding

Publisher's Copyright Statement

Access Output

Other Access Options

Permanent Link

Fingerprint

Cite this