TY - GEN
T1 - Assessing the utility of the system usability scale for evaluating voice-based user interfaces
AU - Ghosh, Debjyoti
AU - Foong, Pin Sym
AU - Zhang, Shan
AU - Zhao, Shengdong
N1 - Publication details (e.g. title, author(s), publication statuses and dates) are captured on an “AS IS” and “AS AVAILABLE” basis at the time of record harvesting from the data source. Suggestions for further amendments or supplementary information can be sent to [email protected].
PY - 2018/4/21
Y1 - 2018/4/21
N2 - Voice-based User Interfaces (VUIs) challenge our existing conceptions of usability since the standardized evaluation tools we use were typically developed for interfaces with visual feedback, whereas VUI's have predominantly eyes-free interactions. We experimented with the use of a well-validated tool, the System Usability Scale (SUS), to evaluate two existing, commercially available VUIs. We administered the SUS to 12 participants after they completed a set of scenario tasks on Amazon's Alexa and Apple's Siri. The results were consistent with previous studies comparing subjective rating scores and SUS, suggesting that the SUS is a valid evaluation tool for VUIs. Additionally, despite large, significant differences in adjective scale ratings and SUS scores, both systems performed similarly on the learnability items. We conclude with recommendations on the use of the SUS for evaluating VUIs. © 2018 Association for Computing Machinery.
AB - Voice-based User Interfaces (VUIs) challenge our existing conceptions of usability since the standardized evaluation tools we use were typically developed for interfaces with visual feedback, whereas VUI's have predominantly eyes-free interactions. We experimented with the use of a well-validated tool, the System Usability Scale (SUS), to evaluate two existing, commercially available VUIs. We administered the SUS to 12 participants after they completed a set of scenario tasks on Amazon's Alexa and Apple's Siri. The results were consistent with previous studies comparing subjective rating scores and SUS, suggesting that the SUS is a valid evaluation tool for VUIs. Additionally, despite large, significant differences in adjective scale ratings and SUS scores, both systems performed similarly on the learnability items. We conclude with recommendations on the use of the SUS for evaluating VUIs. © 2018 Association for Computing Machinery.
KW - Conversational user interface
KW - Evaluation
KW - Voice user interface
KW - Voice-based user interface
UR - http://www.scopus.com/inward/record.url?scp=85049399247&partnerID=8YFLogxK
UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85049399247&origin=recordpage
U2 - 10.1145/3202667.3204844
DO - 10.1145/3202667.3204844
M3 - RGC 32 - Refereed conference paper (with host publication)
SN - 9781450365086
T3 - ACM International Conference Proceeding Series
SP - 11
EP - 15
BT - Proceedings of Chinese CHI 2018 - 6th International Symposium of Chinese CHI, ChineseCHI 2018
PB - Association for Computing Machinery
T2 - 6th International Symposium of Chinese CHI, ChineseCHI 2018
Y2 - 21 April 2018 through 22 April 2018
ER -