An exploratory study on instructors' agreement on the correctness of computer program outputs
Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review
Author(s)
Related Research Unit(s)
Detail(s)
| Original language | English |
|---|---|
| Title of host publication | Hybrid Learning and Continuing Education |
| Subtitle of host publication | 6th International Conference, ICHL 2013, Proceedings |
| Publisher | Springer Verlag |
| Pages | 69-80 |
| Volume | 8038 LNCS |
| ISBN (print) | 9783642397493 |
| Publication status | Published - 2013 |
Publication series
| Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
|---|---|
| Volume | 8038 LNCS |
| ISSN (Print) | 0302-9743 |
| ISSN (electronic) | 1611-3349 |
Conference
| Title | 6th International Conference on Hybrid Learning (ICHL 2013) |
|---|---|
| Location | University of Toronto |
| Place | Canada |
| City | Toronto |
| Period | 12 - 14 August 2013 |
Link(s)
Abstract
Many universities have developed Automated Program Assessment Systems to automate the tasks of assessing students' computer programs so as to enhance students' learning and relieve instructors' workload. These systems typically evaluate the correctness of a program by comparing its actual outputs with the instructor's pre-defined expected outputs. However, an actual output may still be correct even if it deviates from the expected output. One challenge in building such a system is to devise an automated mechanism for determining program output correctness that matches the instructor's own judgment. This is difficult if instructors have different individual judgments. This paper reports an exploratory empirical study which evaluates instructors' agreement on the correctness of students' program outputs. Our study demonstrates reasonably good overall agreement between the instructors and reveals the categories of program output variants for which they are more likely to agree or disagree. © Springer-Verlag 2013.
Research Area(s)
- Automated assessment, Instructor agreement, Output correctness, Program testing, Testing design, Token pattern
Citation Format(s)
An exploratory study on instructors' agreement on the correctness of computer program outputs. / Tang, Chung Man; Yu, Y. T.
Hybrid Learning and Continuing Education: 6th International Conference, ICHL 2013, Proceedings. Vol. 8038 LNCS Springer Verlag, 2013. p. 69-80 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8038 LNCS).
Hybrid Learning and Continuing Education: 6th International Conference, ICHL 2013, Proceedings. Vol. 8038 LNCS Springer Verlag, 2013. p. 69-80 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 8038 LNCS).
Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review
