De-identification of Clinical Text via Bi-LSTM-CRF with Neural Language Models

Buzhou Tang; Dehuan Jiang; Qingcai Chen; Xiaolong Wang; Jun Yan; Ying Shen

De-identification of Clinical Text via Bi-LSTM-CRF with Neural Language Models

Buzhou Tang^*, Dehuan Jiang, Qingcai Chen, Xiaolong Wang, Jun Yan, Ying Shen

^*Corresponding author for this work

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

23 Citations (Scopus)

Abstract

De-identification of clinical text, the prerequisite of electronic clinical data reuse, is a typical named entity recogni tion (NER) problem. A number of state-of-the-art deep learning methods for NER, such as Bi-LSTM-CRF (bidirec tional long-short-term-memory conditional random fields), have been applied for de-identification. Neural language models used for language representation bring great improvement in lots of NLP tasks when they are integrated with other deep learning methods. In this paper, we introduce Bi-LSTM-CRF with neural language models for de- identification of clinical text, and evaluate it on the de-identification datasets of the i2b2 2014 and the CEGS N- GRID 2016 challenges. Four neural language models of three types individually integrated with Bi-LSTM-CRF are compared in this study. Bi-LSTM-CRF with neural language models achieves the highest “strict” micro-averaged F1-score of 95.50% on the i2b2 2014 dataset and 91.82% on the CEGS N-GRID 2016 dataset, becoming new benchmark results on these two datasets respectively. © 2019 AMIA.

Original language	English
Title of host publication	AMIA 2019 Annual Symposium Proceedings
Pages	857-863
Number of pages	7
Publication status	Published - Nov 2019
Externally published	Yes
Event	2019 American Medical Informatics Association Annual Symposium (AMIA 2019) - Washington, United States Duration: 16 Nov 2019 → 20 Nov 2019 https://knowledge.amia.org/change

Conference

Conference	2019 American Medical Informatics Association Annual Symposium (AMIA 2019)
Place	United States
City	Washington
Period	16/11/19 → 20/11/19
Internet address	https://knowledge.amia.org/change

Funding

This paper is supported in part by grants: NSFCs (National Natural Science Foundations of China) (U1813215, 61876052 and 61573118), Special Foundation for Technology Research Program of Guangdong Province (2015B010131010), Strategic Emerging Industry Development Special Funds of Shenzhen (JCYJ20160531192358466), Innovation Fund of Harbin Institute of Technology (HIT.NSRIF.2017052).

Research Keywords

De-identification
Named entity recognition
Bidirectional long-short-term-memory
Conditional random fields
Neural language models

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

@inproceedings{7f063d613fab472295a43b9c0c010016,

title = "De-identification of Clinical Text via Bi-LSTM-CRF with Neural Language Models",

abstract = "De-identification of clinical text, the prerequisite of electronic clinical data reuse, is a typical named entity recogni tion (NER) problem. A number of state-of-the-art deep learning methods for NER, such as Bi-LSTM-CRF (bidirec tional long-short-term-memory conditional random fields), have been applied for de-identification. Neural language models used for language representation bring great improvement in lots of NLP tasks when they are integrated with other deep learning methods. In this paper, we introduce Bi-LSTM-CRF with neural language models for de- identification of clinical text, and evaluate it on the de-identification datasets of the i2b2 2014 and the CEGS N- GRID 2016 challenges. Four neural language models of three types individually integrated with Bi-LSTM-CRF are compared in this study. Bi-LSTM-CRF with neural language models achieves the highest “strict” micro-averaged F1-score of 95.50% on the i2b2 2014 dataset and 91.82% on the CEGS N-GRID 2016 dataset, becoming new benchmark results on these two datasets respectively. {\textcopyright} 2019 AMIA.",

keywords = "De-identification, Named entity recognition, Bidirectional long-short-term-memory, Conditional random fields, Neural language models",

author = "Buzhou Tang and Dehuan Jiang and Qingcai Chen and Xiaolong Wang and Jun Yan and Ying Shen",

year = "2019",

month = nov,

language = "English",

pages = "857--863",

booktitle = "AMIA 2019 Annual Symposium Proceedings",

note = "2019 American Medical Informatics Association Annual Symposium (AMIA 2019) ; Conference date: 16-11-2019 Through 20-11-2019",

url = "https://knowledge.amia.org/change",

}

TY - GEN

T1 - De-identification of Clinical Text via Bi-LSTM-CRF with Neural Language Models

AU - Tang, Buzhou

AU - Jiang, Dehuan

AU - Chen, Qingcai

AU - Wang, Xiaolong

AU - Yan, Jun

AU - Shen, Ying

PY - 2019/11

Y1 - 2019/11

N2 - De-identification of clinical text, the prerequisite of electronic clinical data reuse, is a typical named entity recogni tion (NER) problem. A number of state-of-the-art deep learning methods for NER, such as Bi-LSTM-CRF (bidirec tional long-short-term-memory conditional random fields), have been applied for de-identification. Neural language models used for language representation bring great improvement in lots of NLP tasks when they are integrated with other deep learning methods. In this paper, we introduce Bi-LSTM-CRF with neural language models for de- identification of clinical text, and evaluate it on the de-identification datasets of the i2b2 2014 and the CEGS N- GRID 2016 challenges. Four neural language models of three types individually integrated with Bi-LSTM-CRF are compared in this study. Bi-LSTM-CRF with neural language models achieves the highest “strict” micro-averaged F1-score of 95.50% on the i2b2 2014 dataset and 91.82% on the CEGS N-GRID 2016 dataset, becoming new benchmark results on these two datasets respectively. © 2019 AMIA.

AB - De-identification of clinical text, the prerequisite of electronic clinical data reuse, is a typical named entity recogni tion (NER) problem. A number of state-of-the-art deep learning methods for NER, such as Bi-LSTM-CRF (bidirec tional long-short-term-memory conditional random fields), have been applied for de-identification. Neural language models used for language representation bring great improvement in lots of NLP tasks when they are integrated with other deep learning methods. In this paper, we introduce Bi-LSTM-CRF with neural language models for de- identification of clinical text, and evaluate it on the de-identification datasets of the i2b2 2014 and the CEGS N- GRID 2016 challenges. Four neural language models of three types individually integrated with Bi-LSTM-CRF are compared in this study. Bi-LSTM-CRF with neural language models achieves the highest “strict” micro-averaged F1-score of 95.50% on the i2b2 2014 dataset and 91.82% on the CEGS N-GRID 2016 dataset, becoming new benchmark results on these two datasets respectively. © 2019 AMIA.

KW - De-identification

KW - Named entity recognition

KW - Bidirectional long-short-term-memory

KW - Conditional random fields

KW - Neural language models

UR - http://www.scopus.com/inward/record.url?scp=85083812035&partnerID=8YFLogxK

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85083812035&origin=recordpage

M3 - RGC 32 - Refereed conference paper (with host publication)

C2 - 32308882

SP - 857

EP - 863

BT - AMIA 2019 Annual Symposium Proceedings

T2 - 2019 American Medical Informatics Association Annual Symposium (AMIA 2019)

Y2 - 16 November 2019 through 20 November 2019

ER -

De-identification of Clinical Text via Bi-LSTM-CRF with Neural Language Models

Abstract

Conference

Funding

Research Keywords

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this