DEVELOPING AI LITERACY THROUGH THE USE OF CUSTOMIZED BOTS: STUDENTS' PERCEPTIONS AND PRACTICES

The widespread availability of generative AI has important implications for higher education, potentially transforming: 1) how people read, write, and communicate; and 2) how people teach and learn. There has therefore been a shift in the digital literacy needs of students, especially their 'AI literacy', including knowledge of AI and the ability to use AI critically, ethically, creatively, and in context-sensitive ways. In this presentation, we describe a pedagogical initiative implemented in separate university digital literacy courses in Hong Kong and Singapore. The initiative aimed to enhance personalized learning and AI literacy by having students interact with a customized AI bot (‘ConceptBot’) about course concepts, engaging in a Socratic dialogue with the bot trained to ask questions rather than provide answers. Students selected a learning task related either to digital literacy studies or another aspect of their lives, interacted with the ConceptBot, and evaluated the experience in a written reflection. We evaluate this activity based on: pre- and post- questionnaires measuring AI literacy; written reflections and interview transcripts, analyzed thematically; students’ chat logs. Quantitative findings show some improvement in AI literacy, especially dimensions of Skill and Usage and Affect. Qualitative findings illustrate a range of student perceptions and practices in relation to: bot focus, behaviour, and evaluation; effects on learning; interaction with AI tools, with some students commenting that the ConceptBot is more like a tutor and friend than a tool. The findings suggest divergent assumptions about the role that GenAI bots can and should play in university education.