Learning to differentiate better from worse translations

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)

Abstract

We present a pairwise learning-to-rank approach to machine translation evaluation that learns to differentiate better from worse translations in the context of a given reference. We integrate several layers of linguistic information encapsulated in tree-based structures, making use of both the reference and the system output simultaneously, thus bringing our ranking closer to how humans evaluate translations. Most importantly, instead of deciding upfront which types of features are important, we use the learning framework of preference re-ranking kernels to learn the features automatically. The evaluation results show that learning in the proposed framework yields better correlation with humans than computing the direct similarity over the same type of structures. Also, we show our structural kernel learning (SKL) can be a general framework for MT evaluation, in which syntactic and semantic information can be naturally incorporated.

Original languageEnglish
Title of host publicationEMNLP 2014 - 2014 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference
PublisherAssociation for Computational Linguistics (ACL)
Pages214-220
Number of pages7
ISBN (Electronic)9781937284961
Publication statusPublished - 2014
Event2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014 - Doha, Qatar
Duration: 25 Oct 201429 Oct 2014

Other

Other2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014
CountryQatar
CityDoha
Period25/10/1429/10/14

Fingerprint

Syntactics
Linguistics
Semantics

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Computer Vision and Pattern Recognition
  • Information Systems

Cite this

Guzmán, F., Rayhan Joty, S., Marques, L., Moschitti, A., Nakov, P., & Nicosia, M. (2014). Learning to differentiate better from worse translations. In EMNLP 2014 - 2014 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference (pp. 214-220). Association for Computational Linguistics (ACL).

Learning to differentiate better from worse translations. / Guzmán, Francisco; Rayhan Joty, Shafiq; Marques, Lluis; Moschitti, Alessandro; Nakov, Preslav; Nicosia, Massimo.

EMNLP 2014 - 2014 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference. Association for Computational Linguistics (ACL), 2014. p. 214-220.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Guzmán, F, Rayhan Joty, S, Marques, L, Moschitti, A, Nakov, P & Nicosia, M 2014, Learning to differentiate better from worse translations. in EMNLP 2014 - 2014 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference. Association for Computational Linguistics (ACL), pp. 214-220, 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, Doha, Qatar, 25/10/14.
Guzmán F, Rayhan Joty S, Marques L, Moschitti A, Nakov P, Nicosia M. Learning to differentiate better from worse translations. In EMNLP 2014 - 2014 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference. Association for Computational Linguistics (ACL). 2014. p. 214-220
Guzmán, Francisco ; Rayhan Joty, Shafiq ; Marques, Lluis ; Moschitti, Alessandro ; Nakov, Preslav ; Nicosia, Massimo. / Learning to differentiate better from worse translations. EMNLP 2014 - 2014 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference. Association for Computational Linguistics (ACL), 2014. pp. 214-220
@inproceedings{d652663b8a0849a5840edcaa5e6df204,
title = "Learning to differentiate better from worse translations",
abstract = "We present a pairwise learning-to-rank approach to machine translation evaluation that learns to differentiate better from worse translations in the context of a given reference. We integrate several layers of linguistic information encapsulated in tree-based structures, making use of both the reference and the system output simultaneously, thus bringing our ranking closer to how humans evaluate translations. Most importantly, instead of deciding upfront which types of features are important, we use the learning framework of preference re-ranking kernels to learn the features automatically. The evaluation results show that learning in the proposed framework yields better correlation with humans than computing the direct similarity over the same type of structures. Also, we show our structural kernel learning (SKL) can be a general framework for MT evaluation, in which syntactic and semantic information can be naturally incorporated.",
author = "Francisco Guzm{\'a}n and {Rayhan Joty}, Shafiq and Lluis Marques and Alessandro Moschitti and Preslav Nakov and Massimo Nicosia",
year = "2014",
language = "English",
pages = "214--220",
booktitle = "EMNLP 2014 - 2014 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference",
publisher = "Association for Computational Linguistics (ACL)",

}

TY - GEN

T1 - Learning to differentiate better from worse translations

AU - Guzmán, Francisco

AU - Rayhan Joty, Shafiq

AU - Marques, Lluis

AU - Moschitti, Alessandro

AU - Nakov, Preslav

AU - Nicosia, Massimo

PY - 2014

Y1 - 2014

N2 - We present a pairwise learning-to-rank approach to machine translation evaluation that learns to differentiate better from worse translations in the context of a given reference. We integrate several layers of linguistic information encapsulated in tree-based structures, making use of both the reference and the system output simultaneously, thus bringing our ranking closer to how humans evaluate translations. Most importantly, instead of deciding upfront which types of features are important, we use the learning framework of preference re-ranking kernels to learn the features automatically. The evaluation results show that learning in the proposed framework yields better correlation with humans than computing the direct similarity over the same type of structures. Also, we show our structural kernel learning (SKL) can be a general framework for MT evaluation, in which syntactic and semantic information can be naturally incorporated.

AB - We present a pairwise learning-to-rank approach to machine translation evaluation that learns to differentiate better from worse translations in the context of a given reference. We integrate several layers of linguistic information encapsulated in tree-based structures, making use of both the reference and the system output simultaneously, thus bringing our ranking closer to how humans evaluate translations. Most importantly, instead of deciding upfront which types of features are important, we use the learning framework of preference re-ranking kernels to learn the features automatically. The evaluation results show that learning in the proposed framework yields better correlation with humans than computing the direct similarity over the same type of structures. Also, we show our structural kernel learning (SKL) can be a general framework for MT evaluation, in which syntactic and semantic information can be naturally incorporated.

UR - http://www.scopus.com/inward/record.url?scp=84961288568&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84961288568&partnerID=8YFLogxK

M3 - Conference contribution

SP - 214

EP - 220

BT - EMNLP 2014 - 2014 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference

PB - Association for Computational Linguistics (ACL)

ER -