Advancing Chatbot Capabilities: Computational Methods, Collaborative Decision-Making, and Unknown Intent Detection

Abstract

Recent advancements in artificial intelligence (AI) and deep learning have revolutionized chatbot systems, enabling more sophisticated and context-aware interactions. The rapid integration of large language models (LLMs) has further propelled chatbot advancements, providing uninterrupted services around the clock and minimizing workforce costs. However, critical challenges persist in optimizing their computational techniques, enhancing collaborative decision-making capabilities, and reliably detecting unknown user intents, each representing a fundamental barrier to deploying knowledgeable conversational agents. This dissertation aims to address these gaps through three studies that advance chatbot technologies, including establishing systematic taxonomies of deep learning approaches for business applications, demonstrating how collaborative decision-making mechanisms can enhance LLM performance, and introducing a novel adaptive learning framework for unknown intent detection in healthcare e-services.

The first study addresses the gap in systematically mapping the pre-LLM chatbot landscape (2017-2021) and its relevance to modern architectures, along with the critical analysis of chatbot development approaches and underlying deep learning computational methods in the context of business applications. We first contribute to conceptualizing chatbot architectures and illustrating the technical characteristics of two common structures. Next, we explore common deep learning technologies in chatbot design from the perspective of computational methods and usage. Then, we propose a new framework to classify chatbot construction architectures and differentiate the traditional retrieval-based and generation-based chatbots in terms of the modern pipeline and end-to-end structures. Finally, we highlight future research directions of chatbots to enable researchers to devote their efforts to the most promising research topics and commercial scenarios.

Our second study investigates a post-2022 breakthrough by exploring LLM collaborative potential. It explores whether LLMs could enhance their diagnostic accuracy through interaction, drawing inspiration from the medical professionals’ collaborative decision-making practices in complex cases. An experimental study was conducted in China (September–December 2024) to investigate the impact of LLM-generated reference decisions and source disclosure on LLMs’ diagnostic performance. Theoretically, it introduces a collaborative decision-making paradigm for prompt engineering. We used a Chinese clinical diagnostic task in a controlled comparative design, where three Chinese LLMs interpreted symptoms and conditions based on patient queries. LLMs’ outcomes were evaluated through accuracy and weighted F1 score metrics. ANOVA on diagnostic accuracy scores demonstrated that incorporating LLM-generated decisions as a reference could significantly improve diagnostic outcomes, with source disclosure amplifying this improvement. Practically, our findings underscore the potential of LLM collaboration in healthcare, offering strategies to refine response generation and decision-making across various applications.

The third study aims to facilitate healthcare chatbot performance in detecting unknown intents. Nowadays, effectively handling user queries with unknown intents—stemming from technical bottlenecks and the narrow scope of pre-defined intent categories—remains a significant challenge. Furthermore, the wide variation in users’ consultation needs and levels of medical knowledge further complicates the chatbot’s ability to understand natural human language. Failure to address unknown intents can lead to low-quality user experiences or a significant risk of inappropriate information acquisition. In this study, we contribute to a novel decision-boundary adaptive learning approach to unknown intent detection. To address the challenge of knowledge asymmetry between patients and experts, we first made theoretical contributions through a theory-guided knowledge fusion scheme that integrates multi-view knowledge (including chatbot users, medical experts, and system designers) via representation learning. Unknown intent detection is then accomplished through the transformed representation of each query, with a fully adaptive learning design proposed for intent decision boundary determination. We empirically validated our method using real-world user query data from the Tianchi platform and medical information from the Xunyiwenyao website. Across all tested unknown intent ratios (25%, 50%, and 75%), our multi-view decision-boundary adaptive learning method was proven to outperform all the benchmark models on the metrics of accuracy score, macro F1-score, and macro F1-scores over known intent classes and over the unknown intent class, boosting chatbot safety and reliability.

Date of Award	20 Oct 2025
Original language	English
Awarding Institution	City University of Hong Kong
Supervisor	Yiu Keung Raymond LAU (Supervisor) & Jingjun David XU (Co-supervisor)