Error analysis of statistical machine translation output

David Vilar, Jia Xu, Luis Fernando D'Haro, Hermann Ney

Research output: Contribution to conferencePaperpeer-review

212 Scopus citations

Abstract

Evaluation of automatic translation output is a difficult task. Several performance measures like Word Error Rate, Position Independent Word Error Rate and the BLEU and NIST scores are widely use and provide a useful tool for comparing different systems and to evaluate improvements within a system. However the interpretation of all of these measures is not at all clear, and the identification of the most prominent source of errors in a given system using these measures alone is not possible. Therefore some analysis of the generated translations is needed in order to identify the main problems and to focus the research efforts. This area is however mostly unexplored and few works have dealt with it until now. In this paper we will present a framework for classification of the errors of a machine translation system and we will carry out an error analysis of the system used by the RWTH in the first TC-STAR evaluation.

Original languageEnglish
Pages697-702
Number of pages6
StatePublished - 2006
Event5th International Conference on Language Resources and Evaluation, LREC 2006 - Genoa, Italy
Duration: 22 May 200628 May 2006

Conference

Conference5th International Conference on Language Resources and Evaluation, LREC 2006
Country/TerritoryItaly
CityGenoa
Period22/05/0628/05/06

Fingerprint

Dive into the research topics of 'Error analysis of statistical machine translation output'. Together they form a unique fingerprint.

Cite this