Unsupervised Learning of Word Boundary with Description Length Gain

Chunyu Kit; Yorick Wilks

Unsupervised Learning of Word Boundary with Description Length Gain

Chunyu Kit
, Yorick Wilks

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

54 Citations (Scopus)

Abstract

This paper presents an unsupervised approach to lexical acquisition with the goodness measure description length gain (DLG) formulated following classic information theory within the minimum description length (MDL) paradigm. The learning algorithm seeks for an optimal segmentation of an utterance that maximises the description length gain from the individual segments. The resultant segments show a nice correspondence to lexical items (in particular, words) in a natural language like English. Learning experiments on large-scale corpora (e.g., the Brown corpus) have shown the effectiveness of both the learning algorithm and the goodness measure that guides that learning.

Original language	English
Title of host publication	CoNLL 1999 - Computational Natural Language Learning, Held in cooperation with 9th Conference of the European Chapter of the Association for Computational Linguistics, EACL 1999 - Proceedings of a Workshop
Publisher	Association for Computational Linguistics
Pages	1-6
Publication status	Published - 1999
Event	1999 Workshop on Computational Natural Language Learning, CoNLL 1999 - Bergen, Norway Duration: 12 Jun 1999 → … https://aclanthology.org/W99-07

Publication series

Name	CoNLL 1999 - Computational Natural Language Learning, Held in cooperation with 9th Conference of the European Chapter of the Association for Computational Linguistics, EACL 1999 - Proceedings of a Workshop

Conference

Conference	1999 Workshop on Computational Natural Language Learning, CoNLL 1999
Place	Norway
City	Bergen
Period	12/06/99 → …
Internet address	https://aclanthology.org/W99-07

Bibliographical note

Publication details (e.g. title, author(s), publication statuses and dates) are captured on an “AS IS” and “AS AVAILABLE” basis at the time of record harvesting from the data source. Suggestions for further amendments or supplementary information can be sent to [email protected].

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Kit, C., & Wilks, Y. (1999). Unsupervised Learning of Word Boundary with Description Length Gain. In CoNLL 1999 - Computational Natural Language Learning, Held in cooperation with 9th Conference of the European Chapter of the Association for Computational Linguistics, EACL 1999 - Proceedings of a Workshop (pp. 1-6). (CoNLL 1999 - Computational Natural Language Learning, Held in cooperation with 9th Conference of the European Chapter of the Association for Computational Linguistics, EACL 1999 - Proceedings of a Workshop). Association for Computational Linguistics.

Kit, Chunyu ; Wilks, Yorick. / Unsupervised Learning of Word Boundary with Description Length Gain. CoNLL 1999 - Computational Natural Language Learning, Held in cooperation with 9th Conference of the European Chapter of the Association for Computational Linguistics, EACL 1999 - Proceedings of a Workshop. Association for Computational Linguistics, 1999. pp. 1-6 (CoNLL 1999 - Computational Natural Language Learning, Held in cooperation with 9th Conference of the European Chapter of the Association for Computational Linguistics, EACL 1999 - Proceedings of a Workshop).

@inproceedings{4229402092094ccc94d5ae9521fe8845,

title = "Unsupervised Learning of Word Boundary with Description Length Gain",

abstract = "This paper presents an unsupervised approach to lexical acquisition with the goodness measure description length gain (DLG) formulated following classic information theory within the minimum description length (MDL) paradigm. The learning algorithm seeks for an optimal segmentation of an utterance that maximises the description length gain from the individual segments. The resultant segments show a nice correspondence to lexical items (in particular, words) in a natural language like English. Learning experiments on large-scale corpora (e.g., the Brown corpus) have shown the effectiveness of both the learning algorithm and the goodness measure that guides that learning.",

author = "Chunyu Kit and Yorick Wilks",

note = "Publication details (e.g. title, author(s), publication statuses and dates) are captured on an “AS IS” and “AS AVAILABLE” basis at the time of record harvesting from the data source. Suggestions for further amendments or supplementary information can be sent to [email protected].; 1999 Workshop on Computational Natural Language Learning, CoNLL 1999 ; Conference date: 12-06-1999",

year = "1999",

language = "English",

series = "CoNLL 1999 - Computational Natural Language Learning, Held in cooperation with 9th Conference of the European Chapter of the Association for Computational Linguistics, EACL 1999 - Proceedings of a Workshop",

publisher = "Association for Computational Linguistics",

pages = "1--6",

booktitle = "CoNLL 1999 - Computational Natural Language Learning, Held in cooperation with 9th Conference of the European Chapter of the Association for Computational Linguistics, EACL 1999 - Proceedings of a Workshop",

address = "United States",

url = "https://aclanthology.org/W99-07",

}

Kit, C & Wilks, Y 1999, Unsupervised Learning of Word Boundary with Description Length Gain. in CoNLL 1999 - Computational Natural Language Learning, Held in cooperation with 9th Conference of the European Chapter of the Association for Computational Linguistics, EACL 1999 - Proceedings of a Workshop. CoNLL 1999 - Computational Natural Language Learning, Held in cooperation with 9th Conference of the European Chapter of the Association for Computational Linguistics, EACL 1999 - Proceedings of a Workshop, Association for Computational Linguistics, pp. 1-6, 1999 Workshop on Computational Natural Language Learning, CoNLL 1999, Bergen, Norway, 12/06/99.

Unsupervised Learning of Word Boundary with Description Length Gain. / Kit, Chunyu; Wilks, Yorick.
CoNLL 1999 - Computational Natural Language Learning, Held in cooperation with 9th Conference of the European Chapter of the Association for Computational Linguistics, EACL 1999 - Proceedings of a Workshop. Association for Computational Linguistics, 1999. p. 1-6 (CoNLL 1999 - Computational Natural Language Learning, Held in cooperation with 9th Conference of the European Chapter of the Association for Computational Linguistics, EACL 1999 - Proceedings of a Workshop).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Unsupervised Learning of Word Boundary with Description Length Gain

AU - Kit, Chunyu

AU - Wilks, Yorick

N1 - Publication details (e.g. title, author(s), publication statuses and dates) are captured on an “AS IS” and “AS AVAILABLE” basis at the time of record harvesting from the data source. Suggestions for further amendments or supplementary information can be sent to [email protected].

PY - 1999

Y1 - 1999

N2 - This paper presents an unsupervised approach to lexical acquisition with the goodness measure description length gain (DLG) formulated following classic information theory within the minimum description length (MDL) paradigm. The learning algorithm seeks for an optimal segmentation of an utterance that maximises the description length gain from the individual segments. The resultant segments show a nice correspondence to lexical items (in particular, words) in a natural language like English. Learning experiments on large-scale corpora (e.g., the Brown corpus) have shown the effectiveness of both the learning algorithm and the goodness measure that guides that learning.

AB - This paper presents an unsupervised approach to lexical acquisition with the goodness measure description length gain (DLG) formulated following classic information theory within the minimum description length (MDL) paradigm. The learning algorithm seeks for an optimal segmentation of an utterance that maximises the description length gain from the individual segments. The resultant segments show a nice correspondence to lexical items (in particular, words) in a natural language like English. Learning experiments on large-scale corpora (e.g., the Brown corpus) have shown the effectiveness of both the learning algorithm and the goodness measure that guides that learning.

UR - https://www.scopus.com/pages/publications/85090807987

UR - https://www.scopus.com/pages/publications/85090807987#tab=impact

M3 - RGC 32 - Refereed conference paper (with host publication)

T3 - CoNLL 1999 - Computational Natural Language Learning, Held in cooperation with 9th Conference of the European Chapter of the Association for Computational Linguistics, EACL 1999 - Proceedings of a Workshop

SP - 1

EP - 6

BT - CoNLL 1999 - Computational Natural Language Learning, Held in cooperation with 9th Conference of the European Chapter of the Association for Computational Linguistics, EACL 1999 - Proceedings of a Workshop

PB - Association for Computational Linguistics

T2 - 1999 Workshop on Computational Natural Language Learning, CoNLL 1999

Y2 - 12 June 1999

ER -

Kit C, Wilks Y. Unsupervised Learning of Word Boundary with Description Length Gain. In CoNLL 1999 - Computational Natural Language Learning, Held in cooperation with 9th Conference of the European Chapter of the Association for Computational Linguistics, EACL 1999 - Proceedings of a Workshop. Association for Computational Linguistics. 1999. p. 1-6. (CoNLL 1999 - Computational Natural Language Learning, Held in cooperation with 9th Conference of the European Chapter of the Association for Computational Linguistics, EACL 1999 - Proceedings of a Workshop).

Unsupervised Learning of Word Boundary with Description Length Gain

Abstract

Publication series

Conference

Bibliographical note

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this