A tale about PRO and monsters

Preslav Nakov, Francisco Guzmán, Stephan Vogel

Research output: Chapter in Book/Report/Conference proceedingConference contribution

8 Citations (Scopus)

Abstract

While experimenting with tuning on long sentences, we made an unexpected discovery: that PRO falls victim to monsters - overly long negative examples with very low BLEU+1 scores, which are unsuitable for learning and can cause testing BLEU to drop by several points absolute. We propose several effective ways to address the problem, using length- and BLEU+1-based cut-offs, outlier filters, stochastic sampling, and random acceptance. The best of these fixes not only slay and protect against monsters, but also yield higher stability for PRO as well as improved test-time BLEU scores. Thus, we recommend them to anybody using PRO, monster-believer or not.

Original languageEnglish
Title of host publicationACL 2013 - 51st Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference
PublisherAssociation for Computational Linguistics (ACL)
Pages12-17
Number of pages6
Volume2
ISBN (Print)9781937284510
Publication statusPublished - 1 Jan 2013
Event51st Annual Meeting of the Association for Computational Linguistics, ACL 2013 - Sofia, Bulgaria
Duration: 4 Aug 20139 Aug 2013

Other

Other51st Annual Meeting of the Association for Computational Linguistics, ACL 2013
CountryBulgaria
CitySofia
Period4/8/139/8/13

Fingerprint

acceptance
cause
learning
time
Cut
Acceptance
Outliers
Testing
Sampling
Causes
Believer
Filter
Length
Tuning

ASJC Scopus subject areas

  • Language and Linguistics
  • Linguistics and Language

Cite this

Nakov, P., Guzmán, F., & Vogel, S. (2013). A tale about PRO and monsters. In ACL 2013 - 51st Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Vol. 2, pp. 12-17). Association for Computational Linguistics (ACL).

A tale about PRO and monsters. / Nakov, Preslav; Guzmán, Francisco; Vogel, Stephan.

ACL 2013 - 51st Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference. Vol. 2 Association for Computational Linguistics (ACL), 2013. p. 12-17.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Nakov, P, Guzmán, F & Vogel, S 2013, A tale about PRO and monsters. in ACL 2013 - 51st Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference. vol. 2, Association for Computational Linguistics (ACL), pp. 12-17, 51st Annual Meeting of the Association for Computational Linguistics, ACL 2013, Sofia, Bulgaria, 4/8/13.
Nakov P, Guzmán F, Vogel S. A tale about PRO and monsters. In ACL 2013 - 51st Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference. Vol. 2. Association for Computational Linguistics (ACL). 2013. p. 12-17
Nakov, Preslav ; Guzmán, Francisco ; Vogel, Stephan. / A tale about PRO and monsters. ACL 2013 - 51st Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference. Vol. 2 Association for Computational Linguistics (ACL), 2013. pp. 12-17
@inproceedings{896c8c3f19a7489b8bf58d91ab9aedca,
title = "A tale about PRO and monsters",
abstract = "While experimenting with tuning on long sentences, we made an unexpected discovery: that PRO falls victim to monsters - overly long negative examples with very low BLEU+1 scores, which are unsuitable for learning and can cause testing BLEU to drop by several points absolute. We propose several effective ways to address the problem, using length- and BLEU+1-based cut-offs, outlier filters, stochastic sampling, and random acceptance. The best of these fixes not only slay and protect against monsters, but also yield higher stability for PRO as well as improved test-time BLEU scores. Thus, we recommend them to anybody using PRO, monster-believer or not.",
author = "Preslav Nakov and Francisco Guzm{\'a}n and Stephan Vogel",
year = "2013",
month = "1",
day = "1",
language = "English",
isbn = "9781937284510",
volume = "2",
pages = "12--17",
booktitle = "ACL 2013 - 51st Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference",
publisher = "Association for Computational Linguistics (ACL)",

}

TY - GEN

T1 - A tale about PRO and monsters

AU - Nakov, Preslav

AU - Guzmán, Francisco

AU - Vogel, Stephan

PY - 2013/1/1

Y1 - 2013/1/1

N2 - While experimenting with tuning on long sentences, we made an unexpected discovery: that PRO falls victim to monsters - overly long negative examples with very low BLEU+1 scores, which are unsuitable for learning and can cause testing BLEU to drop by several points absolute. We propose several effective ways to address the problem, using length- and BLEU+1-based cut-offs, outlier filters, stochastic sampling, and random acceptance. The best of these fixes not only slay and protect against monsters, but also yield higher stability for PRO as well as improved test-time BLEU scores. Thus, we recommend them to anybody using PRO, monster-believer or not.

AB - While experimenting with tuning on long sentences, we made an unexpected discovery: that PRO falls victim to monsters - overly long negative examples with very low BLEU+1 scores, which are unsuitable for learning and can cause testing BLEU to drop by several points absolute. We propose several effective ways to address the problem, using length- and BLEU+1-based cut-offs, outlier filters, stochastic sampling, and random acceptance. The best of these fixes not only slay and protect against monsters, but also yield higher stability for PRO as well as improved test-time BLEU scores. Thus, we recommend them to anybody using PRO, monster-believer or not.

UR - http://www.scopus.com/inward/record.url?scp=84907319074&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84907319074&partnerID=8YFLogxK

M3 - Conference contribution

SN - 9781937284510

VL - 2

SP - 12

EP - 17

BT - ACL 2013 - 51st Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference

PB - Association for Computational Linguistics (ACL)

ER -