Maximal match chinese segmentation augmented by resources generated from a very large dictionary for post-processing

Ka-Po Chow; Andy C. Chin; Wing Fu Tsoi

Maximal match chinese segmentation augmented by resources generated from a very large dictionary for post-processing

Ka-Po Chow
, Andy C. Chin
, Wing Fu Tsoi

City University of Hong Kong

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

1 Citation (Scopus)

Abstract

We used a production segmentation system, which draws heavily on a large dictionary derived from processing a large amount (over 150 million Chinese characters) of synchronous textual data gathered from various Chinese speech communities, including Beijing, Hong Kong, Taipei, and others. We run this system in two tracks in the Second International Chinese Word Segmentation Bakeoff, with Backward Maximal Matching (right-to-left) as the primary mechanism. We also explored the use of a number of supplementary features offered by the large dictionary in post-processing, in an attempt to resolve ambiguities and detect unknown words. While the results might not have reached their fullest potential, they nevertheless reinforced the importance and usefulness of a large dictionary as a basis for segmentation, and the implication of following a uniform standard on the segmentation performance on data from various sources. © 2005 SIGHAN@IJCNLP 2005 - 4th SIGHAN Workshop on Chinese Language Processing, Proceedings of the Workshop. All Rights Reserved.

Original language	English
Title of host publication	SIGHAN@IJCNLP 2005 - 4th SIGHAN Workshop on Chinese Language Processing, Proceedings of the Workshop
Publisher	ACL Anthology
Pages	176-179
Publication status	Published - 2005
Event	4th SIGHAN Workshop on Chinese Language Processing at the 2nd International Joint Conference on Natural Language Processing, SIGHAN@IJCNLP 2005 - Jeju Island, Korea, Republic of Duration: 14 Oct 2005 → 15 Oct 2005 https://aclanthology.org/volumes/I05-3/

Publication series

Name	SIGHAN
Publisher	IJCNLP 2005 - 4th SIGHAN Workshop on Chinese Language Processing, Proceedings of the Workshop

Conference

Conference	4th SIGHAN Workshop on Chinese Language Processing at the 2nd International Joint Conference on Natural Language Processing, SIGHAN@IJCNLP 2005
Place	Korea, Republic of
City	Jeju Island
Period	14/10/05 → 15/10/05
Internet address	https://aclanthology.org/volumes/I05-3/

Bibliographical note

Publication details (e.g. title, author(s), publication statuses and dates) are captured on an “AS IS” and “AS AVAILABLE” basis at the time of record harvesting from the data source. Suggestions for further amendments or supplementary information can be sent to [email protected].

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

@inproceedings{c4debc456c78469dbf41d2f03807d15b,

title = "Maximal match chinese segmentation augmented by resources generated from a very large dictionary for post-processing",

abstract = "We used a production segmentation system, which draws heavily on a large dictionary derived from processing a large amount (over 150 million Chinese characters) of synchronous textual data gathered from various Chinese speech communities, including Beijing, Hong Kong, Taipei, and others. We run this system in two tracks in the Second International Chinese Word Segmentation Bakeoff, with Backward Maximal Matching (right-to-left) as the primary mechanism. We also explored the use of a number of supplementary features offered by the large dictionary in post-processing, in an attempt to resolve ambiguities and detect unknown words. While the results might not have reached their fullest potential, they nevertheless reinforced the importance and usefulness of a large dictionary as a basis for segmentation, and the implication of following a uniform standard on the segmentation performance on data from various sources. {\textcopyright} 2005 SIGHAN@IJCNLP 2005 - 4th SIGHAN Workshop on Chinese Language Processing, Proceedings of the Workshop. All Rights Reserved.",

author = "Ka-Po Chow and Chin, \{Andy C.\} and Tsoi, \{Wing Fu\}",

note = "Publication details (e.g. title, author(s), publication statuses and dates) are captured on an “AS IS” and “AS AVAILABLE” basis at the time of record harvesting from the data source. Suggestions for further amendments or supplementary information can be sent to [email protected].; 4th SIGHAN Workshop on Chinese Language Processing at the 2nd International Joint Conference on Natural Language Processing, SIGHAN@IJCNLP 2005 ; Conference date: 14-10-2005 Through 15-10-2005",

year = "2005",

language = "English",

series = "SIGHAN",

publisher = "ACL Anthology",

pages = "176--179",

booktitle = "SIGHAN@IJCNLP 2005 - 4th SIGHAN Workshop on Chinese Language Processing, Proceedings of the Workshop",

address = "Australia",

url = "https://aclanthology.org/volumes/I05-3/",

}

Chow, K-P, Chin, AC & Tsoi, WF 2005, Maximal match chinese segmentation augmented by resources generated from a very large dictionary for post-processing. in SIGHAN@IJCNLP 2005 - 4th SIGHAN Workshop on Chinese Language Processing, Proceedings of the Workshop. SIGHAN, ACL Anthology, pp. 176-179, 4th SIGHAN Workshop on Chinese Language Processing at the 2nd International Joint Conference on Natural Language Processing, SIGHAN@IJCNLP 2005, Jeju Island, Korea, Republic of, 14/10/05.

Maximal match chinese segmentation augmented by resources generated from a very large dictionary for post-processing. / Chow, Ka-Po; Chin, Andy C.; Tsoi, Wing Fu.
SIGHAN@IJCNLP 2005 - 4th SIGHAN Workshop on Chinese Language Processing, Proceedings of the Workshop. ACL Anthology, 2005. p. 176-179 (SIGHAN).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Maximal match chinese segmentation augmented by resources generated from a very large dictionary for post-processing

AU - Chow, Ka-Po

AU - Chin, Andy C.

AU - Tsoi, Wing Fu

N1 - Publication details (e.g. title, author(s), publication statuses and dates) are captured on an “AS IS” and “AS AVAILABLE” basis at the time of record harvesting from the data source. Suggestions for further amendments or supplementary information can be sent to [email protected].

PY - 2005

Y1 - 2005

N2 - We used a production segmentation system, which draws heavily on a large dictionary derived from processing a large amount (over 150 million Chinese characters) of synchronous textual data gathered from various Chinese speech communities, including Beijing, Hong Kong, Taipei, and others. We run this system in two tracks in the Second International Chinese Word Segmentation Bakeoff, with Backward Maximal Matching (right-to-left) as the primary mechanism. We also explored the use of a number of supplementary features offered by the large dictionary in post-processing, in an attempt to resolve ambiguities and detect unknown words. While the results might not have reached their fullest potential, they nevertheless reinforced the importance and usefulness of a large dictionary as a basis for segmentation, and the implication of following a uniform standard on the segmentation performance on data from various sources. © 2005 SIGHAN@IJCNLP 2005 - 4th SIGHAN Workshop on Chinese Language Processing, Proceedings of the Workshop. All Rights Reserved.

AB - We used a production segmentation system, which draws heavily on a large dictionary derived from processing a large amount (over 150 million Chinese characters) of synchronous textual data gathered from various Chinese speech communities, including Beijing, Hong Kong, Taipei, and others. We run this system in two tracks in the Second International Chinese Word Segmentation Bakeoff, with Backward Maximal Matching (right-to-left) as the primary mechanism. We also explored the use of a number of supplementary features offered by the large dictionary in post-processing, in an attempt to resolve ambiguities and detect unknown words. While the results might not have reached their fullest potential, they nevertheless reinforced the importance and usefulness of a large dictionary as a basis for segmentation, and the implication of following a uniform standard on the segmentation performance on data from various sources. © 2005 SIGHAN@IJCNLP 2005 - 4th SIGHAN Workshop on Chinese Language Processing, Proceedings of the Workshop. All Rights Reserved.

UR - https://www.scopus.com/pages/publications/78651436246

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-78651436246&origin=recordpage

M3 - RGC 32 - Refereed conference paper (with host publication)

T3 - SIGHAN

SP - 176

EP - 179

BT - SIGHAN@IJCNLP 2005 - 4th SIGHAN Workshop on Chinese Language Processing, Proceedings of the Workshop

PB - ACL Anthology

T2 - 4th SIGHAN Workshop on Chinese Language Processing at the 2nd International Joint Conference on Natural Language Processing, SIGHAN@IJCNLP 2005

Y2 - 14 October 2005 through 15 October 2005

ER -

Maximal match chinese segmentation augmented by resources generated from a very large dictionary for post-processing

Abstract

Publication series

Conference

Bibliographical note

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this