PIRLS Category-specific Question Generation for Reading Comprehension

Yin Poon; Qiong Wang; John S. Y. Lee; Yu Yan Lam; Samuel Kai Wah Chu

PIRLS Category-specific Question Generation for Reading Comprehension

Yin Poon, Qiong Wang, John S. Y. Lee, Yu Yan Lam, Samuel Kai Wah Chu

Department of Linguistics and Translation

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

15 Downloads (CityUHK Scholars)

Abstract

According to the internationally recognized PIRLS (Progress in International Reading Literacy Study) assessment standards, reading comprehension questions should encompass all four comprehension processes: retrieval, inferencing, integrating and evaluation. This paper investigates whether Large Language Models can produce high-quality questions for each of these categories. Human assessment on a Chinese dataset shows that GPT-4o can generate usable and category-specific questions, ranging from 74% to 90% accuracy depending on the category.

Original language	English
Title of host publication	Proceedings of the 14th Workshop on Natural Language Processing for Computer Assisted Language Learning (NLP4CALL 2025)
Editors	Ricardo Muñoz Sánchez, David Alfter, Elena Volodina, Jelena Kallas
Publisher	University of Tartu Library
Pages	72-80
ISBN (Print)	978-9908-53-112-0
Publication status	Published - Mar 2025
Event	14th Workshop on Natural Language Processing for Computer Assisted Language Learning - Tallinn, Estonia Duration: 5 Mar 2025 → … https://spraakbanken.gu.se/en/research/themes/icall/nlp4call-workshop-series/nlp4call2025

Workshop

Workshop	14th Workshop on Natural Language Processing for Computer Assisted Language Learning
Abbreviated title	NLP4CALL 2025
Country/Territory	Estonia
City	Tallinn
Period	5/03/25 → …
Internet address	https://spraakbanken.gu.se/en/research/themes/icall/nlp4call-workshop-series/nlp4call2025

Funding

We gratefully acknowledge support from the eLearning Ancillary Facilities Programme of the Quality Education Fund (Project “Knowledge Overlord - A Self-sustaining AI Game-based Online Platform to Enhance Student’s Literacy Ability and 21st Century Skills”); and from a Teaching Development Grant at City University of Hong Kong (project 6000834).

Publisher's Copyright Statement

This full text is made available under CC-BY 4.0. https://creativecommons.org/licenses/by/4.0/

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access Output

273322775.pdfFinal Published version, 706 KBLicence: CC BY

https://hdl.handle.net/10062/107171

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Poon, Y., Wang, Q., Lee, J. S. Y., Lam, Y. Y., & Chu, S. K. W. (2025). PIRLS Category-specific Question Generation for Reading Comprehension. In R. Muñoz Sánchez, D. Alfter, E. Volodina, & J. Kallas (Eds.), Proceedings of the 14th Workshop on Natural Language Processing for Computer Assisted Language Learning (NLP4CALL 2025) (pp. 72-80). University of Tartu Library. https://hdl.handle.net/10062/107171

Poon, Yin ; Wang, Qiong ; Lee, John S. Y. et al. / PIRLS Category-specific Question Generation for Reading Comprehension. Proceedings of the 14th Workshop on Natural Language Processing for Computer Assisted Language Learning (NLP4CALL 2025). editor / Ricardo Muñoz Sánchez ; David Alfter ; Elena Volodina ; Jelena Kallas. University of Tartu Library, 2025. pp. 72-80

@inproceedings{35c962f055af415785a42d761efea61e,

title = "PIRLS Category-specific Question Generation for Reading Comprehension",

abstract = "According to the internationally recognized PIRLS (Progress in International Reading Literacy Study) assessment standards, reading comprehension questions should encompass all four comprehension processes: retrieval, inferencing, integrating and evaluation. This paper investigates whether Large Language Models can produce high-quality questions for each of these categories. Human assessment on a Chinese dataset shows that GPT-4o can generate usable and category-specific questions, ranging from 74% to 90% accuracy depending on the category.",

author = "Yin Poon and Qiong Wang and Lee, {John S. Y.} and Lam, {Yu Yan} and Chu, {Samuel Kai Wah}",

year = "2025",

month = mar,

language = "English",

isbn = "978-9908-53-112-0",

pages = "72--80",

editor = "{Mu{\~n}oz S{\'a}nchez}, Ricardo and David Alfter and Elena Volodina and Jelena Kallas",

booktitle = "Proceedings of the 14th Workshop on Natural Language Processing for Computer Assisted Language Learning (NLP4CALL 2025)",

publisher = "University of Tartu Library",

note = "14th Workshop on Natural Language Processing for Computer Assisted Language Learning, NLP4CALL 2025 ; Conference date: 05-03-2025",

url = "https://spraakbanken.gu.se/en/research/themes/icall/nlp4call-workshop-series/nlp4call2025",

}

Poon, Y, Wang, Q, Lee, JSY, Lam, YY & Chu, SKW 2025, PIRLS Category-specific Question Generation for Reading Comprehension. in R Muñoz Sánchez, D Alfter, E Volodina & J Kallas (eds), Proceedings of the 14th Workshop on Natural Language Processing for Computer Assisted Language Learning (NLP4CALL 2025). University of Tartu Library, pp. 72-80, 14th Workshop on Natural Language Processing for Computer Assisted Language Learning, Tallinn, Estonia, 5/03/25. <https://hdl.handle.net/10062/107171>

PIRLS Category-specific Question Generation for Reading Comprehension. / Poon, Yin; Wang, Qiong; Lee, John S. Y. et al.
Proceedings of the 14th Workshop on Natural Language Processing for Computer Assisted Language Learning (NLP4CALL 2025). ed. / Ricardo Muñoz Sánchez; David Alfter; Elena Volodina; Jelena Kallas. University of Tartu Library, 2025. p. 72-80.

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - PIRLS Category-specific Question Generation for Reading Comprehension

AU - Poon, Yin

AU - Wang, Qiong

AU - Lee, John S. Y.

AU - Lam, Yu Yan

AU - Chu, Samuel Kai Wah

PY - 2025/3

Y1 - 2025/3

N2 - According to the internationally recognized PIRLS (Progress in International Reading Literacy Study) assessment standards, reading comprehension questions should encompass all four comprehension processes: retrieval, inferencing, integrating and evaluation. This paper investigates whether Large Language Models can produce high-quality questions for each of these categories. Human assessment on a Chinese dataset shows that GPT-4o can generate usable and category-specific questions, ranging from 74% to 90% accuracy depending on the category.

AB - According to the internationally recognized PIRLS (Progress in International Reading Literacy Study) assessment standards, reading comprehension questions should encompass all four comprehension processes: retrieval, inferencing, integrating and evaluation. This paper investigates whether Large Language Models can produce high-quality questions for each of these categories. Human assessment on a Chinese dataset shows that GPT-4o can generate usable and category-specific questions, ranging from 74% to 90% accuracy depending on the category.

UR - https://spraakbanken.gu.se/en/research/themes/icall/nlp4call-workshop-series/nlp4call2025

UR - https://dspace.ut.ee/items/e7a3cf5b-25e3-4601-84dd-e8357208b683

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 978-9908-53-112-0

SP - 72

EP - 80

BT - Proceedings of the 14th Workshop on Natural Language Processing for Computer Assisted Language Learning (NLP4CALL 2025)

A2 - Muñoz Sánchez, Ricardo

A2 - Alfter, David

A2 - Volodina, Elena

A2 - Kallas, Jelena

PB - University of Tartu Library

T2 - 14th Workshop on Natural Language Processing for Computer Assisted Language Learning

Y2 - 5 March 2025

ER -

PIRLS Category-specific Question Generation for Reading Comprehension

Abstract

Workshop

Funding

Publisher's Copyright Statement

UN SDGs

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

TDG(CityU): Chatbots as Pedagogical Tools: A Feasibility Study

Cite this

PIRLS Category-specific Question Generation for Reading Comprehension

Abstract

Workshop

Funding

Publisher's Copyright Statement

UN SDGs

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Projects

TDG(CityU): Chatbots as Pedagogical Tools: A Feasibility Study

Cite this