DA-BERT : Enhancing Knowledge Selection in Dialog via Domain Adapted BERT with Dynamic Masking Probability

Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45)32_Refereed conference paper (with ISBN/ISSN)peer-review

View graph of relations

Related Research Unit(s)

Detail(s)

Original languageEnglish
Title of host publicationProceedings - 2021 IEEE International Conference on Smart Computing, SMARTCOMP 2021
PublisherInstitute of Electrical and Electronics Engineers
Pages145-152
ISBN (Electronic)978-1-6654-1252-0
ISBN (Print)978-1-6654-2949-8
Publication statusPublished - 2021

Publication series

NameProceedings - IEEE International Conference on Smart Computing, SMARTCOMP
ISSN (Print)2693-8332
ISSN (Electronic)2693-8340

Conference

Title7th IEEE International Conference on Smart Computing, SMARTCOMP 2021
PlaceUnited States
CityVirtual, Irvine
Period23 - 27 August 2021

Abstract

One of the most challenging tasks in Knowledge-grounded Task-oriented Dialog Systems (KTDS) is the knowledge selection task, which aims to find the proper knowledge snippets to handle user requests. This paper proposes DA-BERT to employ pre-trained BERT with domain adaptive training and newly proposed dynamic masking probability to deal with knowledge selection in KTDS. Domain adaptive training minimizes the domain gap between the general text data BERT is pre-trained on and the dialog-knowledge joint data; and dynamic masking probability enhances the training in an easy-to-hard mode. Experimental results on the benchmark dataset show that our proposed training method outperforms the state-of-the-art models with large margins across all the evaluation metrics. Moreover, we analyze the bad case of our method and recognize several typical errors in the bad case set to facilitate further research in this direction.

Research Area(s)

  • Domain adaptive training, dynamic masking probability, knowledge selection, knowledge-grounded task-oriented dialog

Citation Format(s)

DA-BERT : Enhancing Knowledge Selection in Dialog via Domain Adapted BERT with Dynamic Masking Probability. / Zeng, Zhi; Chow, Chi-Yin; Li, Ning.

Proceedings - 2021 IEEE International Conference on Smart Computing, SMARTCOMP 2021. Institute of Electrical and Electronics Engineers, 2021. p. 145-152 (Proceedings - IEEE International Conference on Smart Computing, SMARTCOMP).

Research output: Chapters, Conference Papers, Creative and Literary Works (RGC: 12, 32, 41, 45)32_Refereed conference paper (with ISBN/ISSN)peer-review