Code Debugging with LLM-Generated Explanations of Programming Error Messages

John S. Y. Lee; Fengkai Liu; Tianyuan Cai

doi:10.1109/ICEED62316.2024.10923833

Code Debugging with LLM-Generated Explanations of Programming Error Messages

John S. Y. Lee, Fengkai Liu, Tianyuan Cai

Department of Linguistics and Translation

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

Abstract

Programming is an essential part of the curriculum for electrical, computer and software engineering students. Since one inevitably makes coding mistakes, it is important for programmers to develop debugging skills. However, it could be challenging for beginners to repair a non-compiling program, since programming error messages tend to be opaque, and might not directly address the error. This paper investigates the use of Large Language Models to generate plain, novice-friendly explanations of programming error messages. In an introductory course on Natural Language Processing, We evaluate the extent to which these explanations help students debug Python code with six common programming error categories. Experimental results suggest that explanations generated with zero-shot GPT-4 are effective in raising the code revision success rate. © Copyright 2024 IEEE

Original language	English
Title of host publication	2024 IEEE 13th International Conference on Engineering Education (ICEED)
Subtitle of host publication	Dissemination and Advancement of Engineering Education using Artificial Intelligence
Publisher	IEEE
ISBN (Electronic)	979-8-3503-6741-6
ISBN (Print)	979-8-3503-6742-3
DOIs	https://doi.org/10.1109/ICEED62316.2024.10923833
Publication status	Online published - 20 Mar 2025
Event	13th IEEE International Conference on Engineering Education (ICEED 2024): Dissemination and Advancement of Engineering Education using Artificial Intelligence - Kanazawa, Japan Duration: 19 Nov 2024 → 20 Nov 2024 https://ieeexplore.ieee.org/xpl/conhome/10923724/proceeding?sortType=vol-only-seq&isnumber=10923726&pageNumber=1

Conference

Conference	13th IEEE International Conference on Engineering Education (ICEED 2024)
Abbreviated title	ICEED 2024
Country/Territory	Japan
City	Kanazawa
Period	19/11/24 → 20/11/24
Internet address	https://ieeexplore.ieee.org/xpl/conhome/10923724/proceeding?sortType=vol-only-seq&isnumber=10923726&pageNumber=1

Funding

This work was partly supported by a Teaching Development Grant from City University of Hong Kong (project 6000834).

Research Keywords

computer science
computer engineering
software engineering
Python
debugging
Large Language Model
Programming Error Message

Access Output

10.1109/ICEED62316.2024.10923833

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

@inproceedings{54dd28b8d306474d9f3989655e0a5787,

title = "Code Debugging with LLM-Generated Explanations of Programming Error Messages",

abstract = "Programming is an essential part of the curriculum for electrical, computer and software engineering students. Since one inevitably makes coding mistakes, it is important for programmers to develop debugging skills. However, it could be challenging for beginners to repair a non-compiling program, since programming error messages tend to be opaque, and might not directly address the error. This paper investigates the use of Large Language Models to generate plain, novice-friendly explanations of programming error messages. In an introductory course on Natural Language Processing, We evaluate the extent to which these explanations help students debug Python code with six common programming error categories. Experimental results suggest that explanations generated with zero-shot GPT-4 are effective in raising the code revision success rate. {\textcopyright} Copyright 2024 IEEE ",

keywords = "computer science, computer engineering, software engineering, Python, debugging, Large Language Model, Programming Error Message",

author = "Lee, {John S. Y.} and Fengkai Liu and Tianyuan Cai",

year = "2025",

month = mar,

day = "20",

doi = "10.1109/ICEED62316.2024.10923833",

language = "English",

isbn = "979-8-3503-6742-3",

booktitle = "2024 IEEE 13th International Conference on Engineering Education (ICEED)",

publisher = "IEEE",

address = "United States",

note = "13th IEEE International Conference on Engineering Education (ICEED 2024) : Dissemination and Advancement of Engineering Education using Artificial Intelligence, ICEED 2024 ; Conference date: 19-11-2024 Through 20-11-2024",

url = "https://ieeexplore.ieee.org/xpl/conhome/10923724/proceeding?sortType=vol-only-seq&isnumber=10923726&pageNumber=1",

}

Lee, JSY , Liu, F & Cai, T 2025, Code Debugging with LLM-Generated Explanations of Programming Error Messages. in 2024 IEEE 13th International Conference on Engineering Education (ICEED): Dissemination and Advancement of Engineering Education using Artificial Intelligence. IEEE, 13th IEEE International Conference on Engineering Education (ICEED 2024), Kanazawa, Japan, 19/11/24. https://doi.org/10.1109/ICEED62316.2024.10923833

Code Debugging with LLM-Generated Explanations of Programming Error Messages. / Lee, John S. Y.; Liu, Fengkai ; Cai, Tianyuan.
2024 IEEE 13th International Conference on Engineering Education (ICEED): Dissemination and Advancement of Engineering Education using Artificial Intelligence. IEEE, 2025.

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Code Debugging with LLM-Generated Explanations of Programming Error Messages

AU - Lee, John S. Y.

AU - Liu, Fengkai

AU - Cai, Tianyuan

PY - 2025/3/20

Y1 - 2025/3/20

N2 - Programming is an essential part of the curriculum for electrical, computer and software engineering students. Since one inevitably makes coding mistakes, it is important for programmers to develop debugging skills. However, it could be challenging for beginners to repair a non-compiling program, since programming error messages tend to be opaque, and might not directly address the error. This paper investigates the use of Large Language Models to generate plain, novice-friendly explanations of programming error messages. In an introductory course on Natural Language Processing, We evaluate the extent to which these explanations help students debug Python code with six common programming error categories. Experimental results suggest that explanations generated with zero-shot GPT-4 are effective in raising the code revision success rate. © Copyright 2024 IEEE

AB - Programming is an essential part of the curriculum for electrical, computer and software engineering students. Since one inevitably makes coding mistakes, it is important for programmers to develop debugging skills. However, it could be challenging for beginners to repair a non-compiling program, since programming error messages tend to be opaque, and might not directly address the error. This paper investigates the use of Large Language Models to generate plain, novice-friendly explanations of programming error messages. In an introductory course on Natural Language Processing, We evaluate the extent to which these explanations help students debug Python code with six common programming error categories. Experimental results suggest that explanations generated with zero-shot GPT-4 are effective in raising the code revision success rate. © Copyright 2024 IEEE

KW - computer science

KW - computer engineering

KW - software engineering

KW - Python

KW - debugging

KW - Large Language Model

KW - Programming Error Message

UR - http://www.scopus.com/inward/record.url?scp=105001873911&partnerID=8YFLogxK

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-105001873911&origin=recordpage

U2 - 10.1109/ICEED62316.2024.10923833

DO - 10.1109/ICEED62316.2024.10923833

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 979-8-3503-6742-3

BT - 2024 IEEE 13th International Conference on Engineering Education (ICEED)

PB - IEEE

T2 - 13th IEEE International Conference on Engineering Education (ICEED 2024)

Y2 - 19 November 2024 through 20 November 2024

ER -

Code Debugging with LLM-Generated Explanations of Programming Error Messages

Abstract

Conference

Funding

Research Keywords

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this