Comparing Chinese‐English MT Performance Involving ChatGPT and MT Providers and the Efficacy of AI mediated Post‐Editing

Larry P. Cady; Benjamin K. Tsou; John S. Y. Lee

Comparing Chinese‐English MT Performance Involving ChatGPT and MT Providers and the Efficacy of AI mediated Post‐Editing

Larry P. Cady, Benjamin K. Tsou, John S. Y. Lee

Department of Linguistics and Translation

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

4 Citations (Scopus)

106 Downloads (CityUHK Scholars)

Abstract

The recent introduction of ChatGPT has caused much stir in the translation industry because of its impressive translation performance against leaders in the industry. We review some major issues based on the BLEU comparisons of Chinese-to-English (C2E) and English-toChinese (E2C) machine translation (MT) performance by ChatGPT against a range of leading MT providers in mostly technical domains. Based on sample aligned sentences from a sizable bilingual Chinese-English patent corpus and other sources, we find that while ChatGPT performs better generally, it does not consistently perform better than others in all areas or cases.
We also draw on novice translators as post-editors to explore a major component1 in MT post-editing: Optimization of terminology. Many new technical words, including MWEs (Multi-Word Expressions), are problematic because they involve terminological developments which must balance between proper encapsulation of technical innovation and conforming to past traditions2. Drawing on the above-mentioned reference corpus3 we have been developing an AI mediated MT post-editing (MTPE) system through the optimization of precedent rendition distribution and semantic association to enhance the work of translators and MTPE practitioners. © 2023 The authors.

Original language	English
Title of host publication	Proceedings of Machine Translation Summit XIX
Publisher	Asia-Pacific Association for Machine Translation
Pages	205-216
Volume	2: Users Track
ISBN (Print)	9780000000002
Publication status	Published - Sept 2023
Event	Machine Translation Summit XIX (MT Summit 2023) - Studio City, Macau, China Duration: 4 Sept 2023 → 8 Sept 2023 https://mtsummit2023.scimeeting.cn/en/web/index/15680_ https://aclanthology.org/2023.mtsummit-research

Publication series

Name	MT Summit - Proceedings of Machine Translation Summit
Volume	2 (2023)

Conference

Conference	Machine Translation Summit XIX (MT Summit 2023)
Abbreviated title	MTS 2023
Country/Territory	China
City	Macau
Period	4/09/23 → 8/09/23
Internet address	https://mtsummit2023.scimeeting.cn/en/web/index/15680_ https://aclanthology.org/2023.mtsummit-research

Bibliographical note

Research Unit(s) information for this publication is provided by the author(s) concerned.

Publisher's Copyright Statement

This full text is made available under CC-BY 4.0. https://creativecommons.org/licenses/by/4.0/

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access Output

169316266.pdfFinal Published version, 1.1 MBLicence: CC BY

Other Access Options

Check CityUHK Library

Permanent Link

Right click to copy link

Cite this

Cady, L. P., Tsou, B. K., & Lee, J. S. Y. (2023). Comparing Chinese‐English MT Performance Involving ChatGPT and MT Providers and the Efficacy of AI mediated Post‐Editing. In Proceedings of Machine Translation Summit XIX (Vol. 2: Users Track, pp. 205-216). (MT Summit - Proceedings of Machine Translation Summit; Vol. 2 (2023)). Asia-Pacific Association for Machine Translation.

Cady, Larry P. ; Tsou, Benjamin K. ; Lee, John S. Y. / Comparing Chinese‐English MT Performance Involving ChatGPT and MT Providers and the Efficacy of AI mediated Post‐Editing. Proceedings of Machine Translation Summit XIX. Vol. 2: Users Track Asia-Pacific Association for Machine Translation, 2023. pp. 205-216 (MT Summit - Proceedings of Machine Translation Summit).

@inproceedings{7dbd9f77f76a4dc4bde9ed77684cff87,

title = "Comparing Chinese‐English MT Performance Involving ChatGPT and MT Providers and the Efficacy of AI mediated Post‐Editing",

abstract = "The recent introduction of ChatGPT has caused much stir in the translation industry because of its impressive translation performance against leaders in the industry. We review some major issues based on the BLEU comparisons of Chinese-to-English (C2E) and English-toChinese (E2C) machine translation (MT) performance by ChatGPT against a range of leading MT providers in mostly technical domains. Based on sample aligned sentences from a sizable bilingual Chinese-English patent corpus and other sources, we find that while ChatGPT performs better generally, it does not consistently perform better than others in all areas or cases. We also draw on novice translators as post-editors to explore a major component1 in MT post-editing: Optimization of terminology. Many new technical words, including MWEs (Multi-Word Expressions), are problematic because they involve terminological developments which must balance between proper encapsulation of technical innovation and conforming to past traditions2. Drawing on the above-mentioned reference corpus3 we have been developing an AI mediated MT post-editing (MTPE) system through the optimization of precedent rendition distribution and semantic association to enhance the work of translators and MTPE practitioners. {\textcopyright} 2023 The authors.",

author = "Cady, {Larry P.} and Tsou, {Benjamin K.} and Lee, {John S. Y.}",

note = "Research Unit(s) information for this publication is provided by the author(s) concerned.; Machine Translation Summit XIX (MT Summit 2023), MTS 2023 ; Conference date: 04-09-2023 Through 08-09-2023",

year = "2023",

month = sep,

language = "English",

isbn = "9780000000002",

volume = "2: Users Track",

series = "MT Summit - Proceedings of Machine Translation Summit",

publisher = "Asia-Pacific Association for Machine Translation",

pages = "205--216",

booktitle = "Proceedings of Machine Translation Summit XIX",

url = "https://mtsummit2023.scimeeting.cn/en/web/index/15680_, https://aclanthology.org/2023.mtsummit-research",

}

Cady, LP, Tsou, BK & Lee, JSY 2023, Comparing Chinese‐English MT Performance Involving ChatGPT and MT Providers and the Efficacy of AI mediated Post‐Editing. in Proceedings of Machine Translation Summit XIX. vol. 2: Users Track, MT Summit - Proceedings of Machine Translation Summit, vol. 2 (2023), Asia-Pacific Association for Machine Translation, pp. 205-216, Machine Translation Summit XIX (MT Summit 2023), Macau, China, 4/09/23.

Comparing Chinese‐English MT Performance Involving ChatGPT and MT Providers and the Efficacy of AI mediated Post‐Editing. / Cady, Larry P.; Tsou, Benjamin K.; Lee, John S. Y.
Proceedings of Machine Translation Summit XIX. Vol. 2: Users Track Asia-Pacific Association for Machine Translation, 2023. p. 205-216 (MT Summit - Proceedings of Machine Translation Summit; Vol. 2 (2023)).

Research output: Chapters, Conference Papers, Creative and Literary Works › RGC 32 - Refereed conference paper (with host publication) › peer-review

TY - GEN

T1 - Comparing Chinese‐English MT Performance Involving ChatGPT and MT Providers and the Efficacy of AI mediated Post‐Editing

AU - Cady, Larry P.

AU - Tsou, Benjamin K.

AU - Lee, John S. Y.

N1 - Research Unit(s) information for this publication is provided by the author(s) concerned.

PY - 2023/9

Y1 - 2023/9

N2 - The recent introduction of ChatGPT has caused much stir in the translation industry because of its impressive translation performance against leaders in the industry. We review some major issues based on the BLEU comparisons of Chinese-to-English (C2E) and English-toChinese (E2C) machine translation (MT) performance by ChatGPT against a range of leading MT providers in mostly technical domains. Based on sample aligned sentences from a sizable bilingual Chinese-English patent corpus and other sources, we find that while ChatGPT performs better generally, it does not consistently perform better than others in all areas or cases. We also draw on novice translators as post-editors to explore a major component1 in MT post-editing: Optimization of terminology. Many new technical words, including MWEs (Multi-Word Expressions), are problematic because they involve terminological developments which must balance between proper encapsulation of technical innovation and conforming to past traditions2. Drawing on the above-mentioned reference corpus3 we have been developing an AI mediated MT post-editing (MTPE) system through the optimization of precedent rendition distribution and semantic association to enhance the work of translators and MTPE practitioners. © 2023 The authors.

AB - The recent introduction of ChatGPT has caused much stir in the translation industry because of its impressive translation performance against leaders in the industry. We review some major issues based on the BLEU comparisons of Chinese-to-English (C2E) and English-toChinese (E2C) machine translation (MT) performance by ChatGPT against a range of leading MT providers in mostly technical domains. Based on sample aligned sentences from a sizable bilingual Chinese-English patent corpus and other sources, we find that while ChatGPT performs better generally, it does not consistently perform better than others in all areas or cases. We also draw on novice translators as post-editors to explore a major component1 in MT post-editing: Optimization of terminology. Many new technical words, including MWEs (Multi-Word Expressions), are problematic because they involve terminological developments which must balance between proper encapsulation of technical innovation and conforming to past traditions2. Drawing on the above-mentioned reference corpus3 we have been developing an AI mediated MT post-editing (MTPE) system through the optimization of precedent rendition distribution and semantic association to enhance the work of translators and MTPE practitioners. © 2023 The authors.

UR - http://www.scopus.com/inward/record.url?scp=85185223682&partnerID=8YFLogxK

UR - https://www.scopus.com/record/pubmetrics.uri?eid=2-s2.0-85185223682&origin=recordpage

M3 - RGC 32 - Refereed conference paper (with host publication)

SN - 9780000000002

VL - 2: Users Track

T3 - MT Summit - Proceedings of Machine Translation Summit

SP - 205

EP - 216

BT - Proceedings of Machine Translation Summit XIX

PB - Asia-Pacific Association for Machine Translation

T2 - Machine Translation Summit XIX (MT Summit 2023)

Y2 - 4 September 2023 through 8 September 2023

ER -

Comparing Chinese‐English MT Performance Involving ChatGPT and MT Providers and the Efficacy of AI mediated Post‐Editing

Abstract

Publication series

Conference

Bibliographical note

Publisher's Copyright Statement

UN SDGs

Access Output

Other Access Options

Permanent Link

Other Files and Links

Fingerprint

Cite this