Identifying and Addressing Limitations of AI-generated Scientific Texts: The Lens of Language Features and Communicative Functions

In HK universities, STEM English courses are often provided to help undergraduate students produce important STEM genres (e.g., scientific reports) that fits the expectation of intended audience in their disciplinary communities. Since the release of ChatGPT in 2022, generative AI has had a large influence on English education. While the use of AI tools (e.g., Gemini, Copilot) has brought pedagogical opportunities, it has also brought potential pitfalls. In terms of English language, a common belief is that AI tools have a strong control of language and can generate texts close to those produced by professional writers (Nguyen & Barrot, 2024). However, from the 2023 to the present, cumulative evidence has shown a set of limitations of AI-generated texts, e.g., the reliance of formulaic sequences, the lack of stance and engagement, an overuse of “empty” language, and audience-blind writing style (Jiang & Hyland, 2025). In 2026, as CityU English language courses (including STEM English courses) will be experiencing a pedagogical shift from prohibiting AI usage to allowing AI usage, it is urgent to a) generate students’ critical awareness of the limitations in AI-generated texts and b) develop their ability to address such limitations when they work on the STEM genres. Therefore, the goal of this project is to develop targeted pedagogical materials and instructions, which can help teachers and students in STEM English courses to have a smooth transition for this pedagogical shift.The project will be conducted in the General Education course ‘English for Science (GE2401)’, a compulsory course for all undergraduate students in science majors at CityU. For pedagogical innovation, the PI has built a linguistic model based on student writing at CityU (see Table 1 in the attached picture below), clarifying four linguistic-functional characteristics that students should master to produce scientific texts, for instance “informationally dense summary” and “compressed procedural discourse”. This linguistic model will be built into the course pedagogy to help students work on the Scientific Report (the final course project and assignment). First, according to teachers’ needs and challenges during this pedagogical shift, we will develop a portfolio of pedagogical materials and instructions based on the PI’s linguistic model. Second, students will receive awareness-raising instructions, clarifying how a specific communicative function (e.g., “compressed procedural discourse”) is achieved through the use certain language features in model samples of scientific reports. Third, students will work on critical evaluation of AI-generated samples of scientific reports, critically identifying their strengths and limitations based on the four linguistic-functional characteristics in the linguistic model. Fourth, students will be guided to address the identified limitations of AI-generated samples, particularly revising the language use associated with certain linguistic-functional characteristics. Last, the portfolio of pedagogical materials will be further tailored and shared with the GE2401 course team for future use. Upon completion of the project questionnaires will be sent to the teachers and students to collect their feedback. This project also has the potential to be replicated in other discipline-specific English courses, such as English for Engineering (GE2410) and English for Business (GE2402).