Eyes don't lie: Predicting machine translation quality using eye movement

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Poorly translated text is often disfluent and difficult to read. In contrast, well-formed translations require less time to process. In this paper, we model the differences in reading patterns of Machine Translation (MT) evaluators using novel features extracted from their gaze data, and we learn to predict the quality scores given by those evaluators. We test our predictions in a pairwise ranking scenario, measuring Kendall's tau correlation with the judgments. We show that our features provide information beyond fluency, and can be combined with BLEU for better predictions. Furthermore, our results show that reading patterns can be used to build semi-automatic metrics that anticipate the scores given by the evaluators.

Original languageEnglish
Title of host publication2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016 - Proceedings of the Conference
PublisherAssociation for Computational Linguistics (ACL)
Pages1082-1088
Number of pages7
ISBN (Electronic)9781941643914
Publication statusPublished - 2016
Event15th Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016 - San Diego, United States
Duration: 12 Jun 201617 Jun 2016

Other

Other15th Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016
CountryUnited States
CitySan Diego
Period12/6/1617/6/16

    Fingerprint

ASJC Scopus subject areas

  • Computer Science Applications
  • Linguistics and Language
  • Language and Linguistics

Cite this

Sajjad, H., Guzmán, F., Durrani, N., Abdelali, A., Bouamor, H., Temnikova, I., & Vogel, S. (2016). Eyes don't lie: Predicting machine translation quality using eye movement. In 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016 - Proceedings of the Conference (pp. 1082-1088). Association for Computational Linguistics (ACL).