Evaluation in machine translation and computer-aided translation

This chapter introduces the key issues and basic principles of MT/CAT evaluation, concerning MT systems with or without human intervention for finalizing translation output. It begins with a brief review of the history of MT/CAT evaluation to outline the evolution of evaluation methodology and technology, along with the development of MT/CAT over the past several decades. The highly context-dependent multi-dimensional nature of the evaluation will then be described, including various applications of system output and different evaluation purposes. Then the existing evaluation methodologies will be presented and illustrated. On the one hand, an MT/CAT system is evaluated as a piece of software in terms of general parameters such as speed and number of supported file formats, subject to existing standards and criteria. On the other hand, its evaluation becomes a matter of text quality assessment because MT outputs are essentially in the form of text. The major approaches of MT evaluation, including both manual and automatic, will be presented with discussion of their strengths and weaknesses.