Effect of chatbot‑assisted language learning: A meta‑analysis

Shunan Zhang; Cheng Shan; John Sie Yuen Lee; ShaoPeng Che; Jang Hyun Kim

doi:10.1007/s10639-023-11805-6

Effect of chatbot‑assisted language learning: A meta‑analysis

Shunan Zhang
, Cheng Shan
, John Sie Yuen Lee
, ShaoPeng Che
, Jang Hyun Kim^*

^*Corresponding author for this work

Department of Linguistics and Translation

Research output: Journal Publications and Reviews › RGC 21 - Publication in refereed journal › peer-review

81 Citations (Scopus)

Abstract

Chatbots have shown great potential for language learning. However, previous studies have reported mixed results on the efficiency of chatbot-assisted language learning (CALL). This study integrated the results of previous experimental studies on CALL by using meta-analysis to explore its effectiveness. A total of 61 samples from 18 studies were examined. The results showed that CALL had a moderate average effect (g = .527). In addition, nine potential moderating variables (educational level, target language, language domain, learning outcome, instruction duration, chatbot interface, chatbot development, task dominance, and interaction way) were identified and discussed. The results of this study provided insights into the use and design of chatbots for language learning.

© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2023

Original language	English
Pages (from-to)	15223–15243
Journal	Education and Information Technologies
Volume	28
Issue number	11
Online published	26 Apr 2023
DOIs	https://doi.org/10.1007/s10639-023-11805-6
Publication status	Published - Nov 2023

Research Keywords

Chatbots
Language Learning
Meta-analysis

Access Output

10.1007/s10639-023-11805-6

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

@article{0d2a712dab7448a8afcc5353cfc062d0,

title = "Effect of chatbot‑assisted language learning: A meta‑analysis",

abstract = "Chatbots have shown great potential for language learning. However, previous studies have reported mixed results on the efficiency of chatbot-assisted language learning (CALL). This study integrated the results of previous experimental studies on CALL by using meta-analysis to explore its effectiveness. A total of 61 samples from 18 studies were examined. The results showed that CALL had a moderate average effect (g = .527). In addition, nine potential moderating variables (educational level, target language, language domain, learning outcome, instruction duration, chatbot interface, chatbot development, task dominance, and interaction way) were identified and discussed. The results of this study provided insights into the use and design of chatbots for language learning. {\textcopyright} The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2023",

keywords = "Chatbots, Language Learning, Meta-analysis",

author = "Shunan Zhang and Cheng Shan and Lee, \{John Sie Yuen\} and ShaoPeng Che and Kim, \{Jang Hyun\}",

year = "2023",

month = nov,

doi = "10.1007/s10639-023-11805-6",

language = "English",

volume = "28",

pages = "15223–15243",

journal = "Education and Information Technologies",

issn = "1360-2357",

publisher = "Springer Nature",

number = "11",

}

TY - JOUR

T1 - Effect of chatbot‑assisted language learning

T2 - A meta‑analysis

AU - Zhang, Shunan

AU - Shan, Cheng

AU - Lee, John Sie Yuen

AU - Che, ShaoPeng

AU - Kim, Jang Hyun

PY - 2023/11

Y1 - 2023/11

N2 - Chatbots have shown great potential for language learning. However, previous studies have reported mixed results on the efficiency of chatbot-assisted language learning (CALL). This study integrated the results of previous experimental studies on CALL by using meta-analysis to explore its effectiveness. A total of 61 samples from 18 studies were examined. The results showed that CALL had a moderate average effect (g = .527). In addition, nine potential moderating variables (educational level, target language, language domain, learning outcome, instruction duration, chatbot interface, chatbot development, task dominance, and interaction way) were identified and discussed. The results of this study provided insights into the use and design of chatbots for language learning. © The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2023

AB - Chatbots have shown great potential for language learning. However, previous studies have reported mixed results on the efficiency of chatbot-assisted language learning (CALL). This study integrated the results of previous experimental studies on CALL by using meta-analysis to explore its effectiveness. A total of 61 samples from 18 studies were examined. The results showed that CALL had a moderate average effect (g = .527). In addition, nine potential moderating variables (educational level, target language, language domain, learning outcome, instruction duration, chatbot interface, chatbot development, task dominance, and interaction way) were identified and discussed. The results of this study provided insights into the use and design of chatbots for language learning. © The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2023

KW - Chatbots

KW - Language Learning

KW - Meta-analysis

UR - https://www.scopus.com/pages/publications/85153846020

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85153846020&origin=recordpage

U2 - 10.1007/s10639-023-11805-6

DO - 10.1007/s10639-023-11805-6

M3 - RGC 21 - Publication in refereed journal

SN - 1360-2357

VL - 28

SP - 15223

EP - 15243

JO - Education and Information Technologies

JF - Education and Information Technologies

IS - 11

ER -