Meta-evaluation of machine translation using parallel legal texts

Billy Tak-Ming Wong; Chunyu Kit

doi:10.1007/978-3-642-00831-3_33

Meta-evaluation of machine translation using parallel legal texts

Billy Tak-Ming Wong
, Chunyu Kit

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

Abstract

In this paper we report our recent work on the evaluation of a number of popular automatic evaluation metrics for machine translation using parallel legal texts. The evaluation is carried out, following a recognized evaluation protocol, to assess the reliability, the strengths and weaknesses of these evaluation metrics in terms of their correlation with human judgment of translation quality. The evaluation results confirm the reliability of the well-known evaluation metrics, BLEU and NIST for English-to-Chinese translation, and also show that our evaluation metric ATEC outperforms all others for Chinese-to-English translation. We also demonstrate the remarkable impact of different evaluation metrics on the ranking of online machine translation systems for legal translation. © 2009 Springer Berlin Heidelberg.

Original language	English
Title of host publication	Computer Processing of Oriental Languages: Language Technology for the Knowledge-based Economy
Subtitle of host publication	22nd International Conference, ICCPOL 2009, Proceedings
Publisher	Springer Verlag
Pages	337-344
Volume	5459 LNAI
ISBN (Print)	3642008305, 9783642008306
DOIs	https://doi.org/10.1007/978-3-642-00831-3_33
Publication status	Published - 2009
Event	22nd International Conference on Computer Processing of Oriental Languages, ICCPOL 2009 - , Hong Kong, China Duration: 26 Mar 2009 → 27 Mar 2009

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	5459 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	22nd International Conference on Computer Processing of Oriental Languages, ICCPOL 2009
Place	Hong Kong, China
Period	26/03/09 → 27/03/09

Research Keywords

ATEC
BLEU
BLIS
Legal Text
Machine Translation Evaluation

Access Output

10.1007/978-3-642-00831-3_33

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Wong, B. T.-M., & Kit, C. (2009). Meta-evaluation of machine translation using parallel legal texts. In Computer Processing of Oriental Languages: Language Technology for the Knowledge-based Economy: 22nd International Conference, ICCPOL 2009, Proceedings (Vol. 5459 LNAI, pp. 337-344). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5459 LNAI). Springer Verlag. https://doi.org/10.1007/978-3-642-00831-3_33

Wong, Billy Tak-Ming ; Kit, Chunyu. / Meta-evaluation of machine translation using parallel legal texts. Computer Processing of Oriental Languages: Language Technology for the Knowledge-based Economy: 22nd International Conference, ICCPOL 2009, Proceedings. Vol. 5459 LNAI Springer Verlag, 2009. pp. 337-344 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{a62f17bf2c2a478caf8f9da3c0281f9e,

title = "Meta-evaluation of machine translation using parallel legal texts",

abstract = "In this paper we report our recent work on the evaluation of a number of popular automatic evaluation metrics for machine translation using parallel legal texts. The evaluation is carried out, following a recognized evaluation protocol, to assess the reliability, the strengths and weaknesses of these evaluation metrics in terms of their correlation with human judgment of translation quality. The evaluation results confirm the reliability of the well-known evaluation metrics, BLEU and NIST for English-to-Chinese translation, and also show that our evaluation metric ATEC outperforms all others for Chinese-to-English translation. We also demonstrate the remarkable impact of different evaluation metrics on the ranking of online machine translation systems for legal translation. {\textcopyright} 2009 Springer Berlin Heidelberg.",

keywords = "ATEC, BLEU, BLIS, Legal Text, Machine Translation Evaluation",

author = "Wong, \{Billy Tak-Ming\} and Chunyu Kit",

year = "2009",

doi = "10.1007/978-3-642-00831-3\_33",

language = "English",

isbn = "3642008305",

volume = "5459 LNAI",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "337--344",

booktitle = "Computer Processing of Oriental Languages: Language Technology for the Knowledge-based Economy",

address = "Germany",

note = "22nd International Conference on Computer Processing of Oriental Languages, ICCPOL 2009 ; Conference date: 26-03-2009 Through 27-03-2009",

}

Wong, BT-M & Kit, C 2009, Meta-evaluation of machine translation using parallel legal texts. in Computer Processing of Oriental Languages: Language Technology for the Knowledge-based Economy: 22nd International Conference, ICCPOL 2009, Proceedings. vol. 5459 LNAI, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 5459 LNAI, Springer Verlag, pp. 337-344, 22nd International Conference on Computer Processing of Oriental Languages, ICCPOL 2009, Hong Kong, China, 26/03/09. https://doi.org/10.1007/978-3-642-00831-3_33

Meta-evaluation of machine translation using parallel legal texts. / Wong, Billy Tak-Ming; Kit, Chunyu.
Computer Processing of Oriental Languages: Language Technology for the Knowledge-based Economy: 22nd International Conference, ICCPOL 2009, Proceedings. Vol. 5459 LNAI Springer Verlag, 2009. p. 337-344 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5459 LNAI).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Meta-evaluation of machine translation using parallel legal texts

AU - Wong, Billy Tak-Ming

AU - Kit, Chunyu

PY - 2009

Y1 - 2009

N2 - In this paper we report our recent work on the evaluation of a number of popular automatic evaluation metrics for machine translation using parallel legal texts. The evaluation is carried out, following a recognized evaluation protocol, to assess the reliability, the strengths and weaknesses of these evaluation metrics in terms of their correlation with human judgment of translation quality. The evaluation results confirm the reliability of the well-known evaluation metrics, BLEU and NIST for English-to-Chinese translation, and also show that our evaluation metric ATEC outperforms all others for Chinese-to-English translation. We also demonstrate the remarkable impact of different evaluation metrics on the ranking of online machine translation systems for legal translation. © 2009 Springer Berlin Heidelberg.

AB - In this paper we report our recent work on the evaluation of a number of popular automatic evaluation metrics for machine translation using parallel legal texts. The evaluation is carried out, following a recognized evaluation protocol, to assess the reliability, the strengths and weaknesses of these evaluation metrics in terms of their correlation with human judgment of translation quality. The evaluation results confirm the reliability of the well-known evaluation metrics, BLEU and NIST for English-to-Chinese translation, and also show that our evaluation metric ATEC outperforms all others for Chinese-to-English translation. We also demonstrate the remarkable impact of different evaluation metrics on the ranking of online machine translation systems for legal translation. © 2009 Springer Berlin Heidelberg.

KW - ATEC

KW - BLEU

KW - BLIS

KW - Legal Text

KW - Machine Translation Evaluation

UR - https://www.scopus.com/pages/publications/70350663031

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-70350663031&origin=recordpage

U2 - 10.1007/978-3-642-00831-3_33

DO - 10.1007/978-3-642-00831-3_33

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 3642008305

SN - 9783642008306

VL - 5459 LNAI

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 337

EP - 344

BT - Computer Processing of Oriental Languages: Language Technology for the Knowledge-based Economy

PB - Springer Verlag

T2 - 22nd International Conference on Computer Processing of Oriental Languages, ICCPOL 2009

Y2 - 26 March 2009 through 27 March 2009

ER -

Wong BTM, Kit C. Meta-evaluation of machine translation using parallel legal texts. In Computer Processing of Oriental Languages: Language Technology for the Knowledge-based Economy: 22nd International Conference, ICCPOL 2009, Proceedings. Vol. 5459 LNAI. Springer Verlag. 2009. p. 337-344. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-642-00831-3_33

Meta-evaluation of machine translation using parallel legal texts

Abstract

Publication series

Conference

Research Keywords

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this