What and How to Reply to Users When Al-Empowered Voice User Interfaces Fail to Execute a Task: An Experimental Study of the Effects of Explanations and Humor

Abstract

Voice user interfaces (VUIs) empowered by artificial intelligence (AI) are being increasingly utilized in smart products. However, many users do not know exactly the task boundary of smart products and often request tasks that are beyond the function scope of these products. As such, VUIs often fail to execute tasks requested by users (i.e., task failures), although many task failures are not caused by VUIs but by users and environmental factors. Nevertheless, users are usually concerned about whether the VUI has the ability and expertise to execute their tasks (i.e., competence belief). Users are also concerned about whether they can depend on VUIs in the future (i.e., trust). VUIs' proper responses to task failures are critical to the success of smart products. This poses a fundamental question: what and how should a VUI respond to users in case of task failures?

Explaining the cause of service failures is a common response used by service providers. This thesis focuses on what and how a VUI should explain the task failure to users to ensure their positive evaluations of the VUI when facing a task failure. Specifically, the first research objective is to examine what a VUI should explain to users to ensure their competence belief in the VUI. The second research objective is to examine how a VUI should explain the task failure to users to further maintain their trust in the VUI.

Study 1 aims to examine how the content of explanations given by a VUI influences users' competence belief in the VUI. Many task failures are not caused by VUIs but by users and environmental factors. However, whether an explanation for different causes of task failures (i.e., VUI, user, and environmental factor) is always effective in dealing with users' competence concerns about the VUI remains unclear. Information Systems (IS) literature mainly focuses on providing explanations regarding the inner workings of information systems (e.g., recommendation systems) to users when the systems work well. Limited research has studied the effects of explanations regarding the cause of service failures. Meanwhile, extant research on service failures has inadequately investigated service failures caused by users. Study 1 focuses on the content of explanations in case of task failures from an attributional perspective. Attribution is the cause of the event, which is critical to understanding the characteristics of the cause of task failures.

Study 1 conducts a series of experiments to test proposed hypotheses. By applying the perspective of psychological contract violation (PCV) and knowledge-based trust, Study 1 first investigates the effects of an explanation on users' competence belief in the VUI. Results reveal that an explanation increases users' competence belief in the VUI through perceived PCV and perceived transparency. Drawing on attribution theory, Study 1 then examines the effects of an explanation with locus (i.e., VUI, users, and environmental factors) and stability (i.e., unstable and stable) attributions on users' competence belief in the VUI. Results show that users' competence belief in the VUI is lower for an explanation with a VUI attribution than that of an explanation with an environment attribution. This is because an environment attribution implies that the task failure is not directly related to the VUI, resulting in higher competence belief in the VUI. In addition, the effect of an explanation across locus attributions (i.e., VUI vs. user) on users' competence belief in the VUI is contingent upon the stability of task failures. Specifically, users' competence belief in the VUI is lower for an explanation with a VUI attribution than that of an explanation with a user attribution when task failures are unstable, but this difference is non-existent when task failures are stable.

Results attest that perceived PCV mediated the effect of an explanation across different locus attributions (i.e., VUI vs. environment) on users' competence belief in the VUI for task failures attributed to a stable cause. However, perceived PCV did not mediate the effect of an explanation across different locus attributions (i.e., VUI vs. user; VUI vs. environment) on users' competence belief in the VUI for task failures attributed to an unstable cause. A follow-up online experiment was conducted to test and confirm an alternative account (i.e., perceived immediacy of resolving task failures) for the effect of an explanation across different locus attributions (i.e., VUI vs. user; VUI vs. environment) on users' competence belief in VUIs under unstable task failures.

Extending the findings from Study 1, Study 2 aims to examine how the rhetoric of explanations given by a VUI influences users' trust in the VUI. When facing task failures, users often become frustrated, and without a proper response from a VUI, they may not think that the VUI "cares" about them. Merely presenting the content of explanations for task failures may not effectively address users' negative emotions or demonstrate sufficient care towards them. Not only the content of explanations but also how a VUI presents the explanations to users will matter for task failures. However, extant IS research mainly focuses on the content of explanations for task failures, while neglecting how to present the explanations for task failures (i.e., the rhetoric of explanations).

Study 2 aims to examine how the rhetoric of explanations given by a VUI influences users' trust in the VUI. When task failures happen, users may hesitate to delegate the task to VUIs in the future (i.e., trust). Specifically, users are concerned about whether VUIs can be relied on for future tasks (i.e., cognitive trust). Moreover, users are concerned about whether VUIs are supportive partners (i.e., affective trust). Study 2 focuses on a commonly used rhetoric of an explanation—humor in VUIs, which is believed to be an effective approach for addressing "embarrassing" situations (e.g., a task failure). Humor can amuse users, lighten the mood, and may strengthen relationships with users. However, considering that VUIs are primarily used to perform tasks, users may perceive VUIs' expressed humor in case of task failures as an attempt to evade responsibility. The central point of Study 2 is that the effect of a VUI's expressed humor in its explanations on users' cognitive trust and affective trust in the VUI is contingent upon situational factors. Specifically, Study 2 considers user characteristic (i.e., users' relationship norm orientation) and humo r characteristic (i.e., the humor type) as the contingent factors that determine the effects of a VUI's expressed humor in its explanations on users' cognitive and affective trust in the VUI.

Study 2 conducts a series of experiments to test proposed hypotheses. Drawing on the literature on user-AI relationships, Study 2 first examines the moderating effect of relationship norm orientation (i.e., functional–tool oriented — treating VUIs as functional tools, or social–actor oriented — treating VUIs as social actors) to delineate when a VUIs' expressed humor in its explanations for task failures influences users' cognitive and affective trust in the VUI. Results reveal that a VUI's expressed humor decreases cognitive trust for functional-tool oriented users whereas it does not affect cognitive trust for social-actor oriented users. Meanwhile, the VUI's expressed humor increases affective trust for social-actor oriented users but not for functional-tool oriented users.

Drawing on the literature on humor, Study 2 then investigates the moderating effect of the humor type (i.e., self–defeating — putting oneself down to seek others' likability vs. affiliative — saying funny things to enhance relationships with others) to explicate what type of humor will strengthen or weaken the effect of a VUI's expressed humor in its explanation on users' cognitive trust and affective trust in the VUI. Results show that the negative effect of humor on cognitive trust is stronger for self-defeating humor than for affiliative humor; by contrast, the positive effect of humor on affective trust is stronger for affiliative humor than for self-defeating humor.

This thesis makes several key contributions. First, this thesis contributes to the information systems literature by extending the scope of research on explanations to an understudied research domain in which the systems explain to users when the systems encounter task failures. It also advances the literature on the explanation by delineating the effects of attributional characteristics of an explanation (i.e., locus and stability attributions) on users' competence belief and their underlying psychological mechanisms (i.e., perceived PCV, perceived transparency, and perceived immediacy of resolving task failures). Second, this thesis contributes to the research on explanation in the IS field by examining the rhetorical aspect of an explanation (i.e., humor) in case of task failures. It also advances the research on humor by explicating the boundary conditions (i.e., relationship norm orientation and humor type) of the effects of a VUI's humor on users' cognitive and affective trust in the VUI, and their psychological mechanisms (i.e., perceived relief and perceived norm violation). This thesis also offers implications for designing VUIs to deal with task failures.

Date of Award	15 May 2025
Original language	English
Awarding Institution	City University of Hong Kong
Supervisor	Yaobin Lu (External Supervisor), Jingjun David XU (Supervisor) & Weiquan WANG (Supervisor)