BiCalBERT: An Efficient Transformer-based Model for Chinese Question Answering

The exponentially growing content on the Internet includes online publications, scientific news, and other expert websites. It brings a formidable challenge to extracting pertinent answers from such vast information. We proposed a deep neural network model based on BiLSTM(Bi-directional Long Short-Term Memory) and ALBERT(A Lite BERT for Self-Supervised Learning of Language Representations) for Chinese question-answering in the scientific context. Our model significantly enhances the generalization capabilities of the transformer-based model for question answering. The character extraction and word embedding modules are designed for tackling intricate science-related queries, swiftly assimilating knowledge from cutting-edge scientific literature, and contributing to constructing a comprehensive scientific knowledge graph. Our model is characterized by compactness, swift execution, and satisfactory accuracy. It has been meticulously fine-tuned, emphasizing multi-sentence coherence augmented by attention mechanisms, thereby ensuring robust scalability. Empirical evaluations on the LCQMC and XNLI datasets demonstrate that our approach surpasses the performance results of BERT and ALBERT, showing the potential for large-scale Chinese question-matching problems. © 2024 Copyright held by the owner/author(s). Publication rights licensed to ACM.