Incorporating Domain Knowledge Repository into Large Language Models for Question Answering of Construction Laws and Regulations in China

The smart question-answering of construction laws and regulations (QAoCLR) could significantly reduce the CLR query cost. Most existing QAoCLR studies rely heavily on conventional small-scale deep learning models (e.g., BERT), resulting in inadequate language understanding and subpar QA performance. Despite large language models (LLMs) showing impressive QA potential in general fields, they remain ill-performed in professional areas (e.g., CLR) due to the lack of domain-specific knowledge incorporation. Hence, our study integrates a domain knowledge repository with LLMs for QAoCLR. It involves (1) creating a repository of 275 filtered laws and regulations, (2) developing a QAoCLR-oriented test data set of 1960 multiple-choice questions from the Chinese Registered Constructor Examinations, (3) retrieving relevant knowledge vectors and incorporating them into LLMs (i.e., GPT-3.5, GPT-4.0, textdavinci-003, and ChatGLM2-6B), and (4) comparing the performance of LLMs with and without domain knowledge incorporation. The results show that LLMs with domain knowledge exceed original LLMs in QAoCLR accuracy by an average of 27.95%. Specifically, there is a 21.25% improvement in single-answer questions and a 44.62% enhancement in multi-answer questions. This work reveals the effectiveness and necessity of domain knowledge incorporation into LLMs for QAoCLR. © ASCE.