A tale about PRO and monsters

Preslav Nakov, Francisco Guzmán, Stephan Vogel

Research output: Chapter in Book/Report/Conference proceedingConference contribution

9 Citations (Scopus)

Abstract

While experimenting with tuning on long sentences, we made an unexpected discovery: that PRO falls victim to monsters - overly long negative examples with very low BLEU+1 scores, which are unsuitable for learning and can cause testing BLEU to drop by several points absolute. We propose several effective ways to address the problem, using length- and BLEU+1-based cut-offs, outlier filters, stochastic sampling, and random acceptance. The best of these fixes not only slay and protect against monsters, but also yield higher stability for PRO as well as improved test-time BLEU scores. Thus, we recommend them to anybody using PRO, monster-believer or not.

Original languageEnglish
Title of host publicationACL 2013 - 51st Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference
PublisherAssociation for Computational Linguistics (ACL)
Pages12-17
Number of pages6
Volume2
ISBN (Print)9781937284510
Publication statusPublished - 1 Jan 2013
Event51st Annual Meeting of the Association for Computational Linguistics, ACL 2013 - Sofia, Bulgaria
Duration: 4 Aug 20139 Aug 2013

Other

Other51st Annual Meeting of the Association for Computational Linguistics, ACL 2013
CountryBulgaria
CitySofia
Period4/8/139/8/13

    Fingerprint

ASJC Scopus subject areas

  • Language and Linguistics
  • Linguistics and Language

Cite this

Nakov, P., Guzmán, F., & Vogel, S. (2013). A tale about PRO and monsters. In ACL 2013 - 51st Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Vol. 2, pp. 12-17). Association for Computational Linguistics (ACL).