Structural relationships for large-scale learning of answer re-ranking

Aliaksei Severyn, Alessandro Moschitti

Research output: Chapter in Book/Report/Conference proceedingConference contribution

59 Citations (Scopus)

Abstract

Supervised learning applied to answer re-ranking can highly improve on the overall accuracy of question answering (QA) systems. The key aspect is that the relationships and properties of the question/answer pair composed of a question and the supporting passage of an answer candidate, can be efficiently compared with those captured by the learnt model. In this paper, we define novel supervised approaches that exploit structural relationships between a question and their candidate answer passages to learn a re-ranking model. We model structural representations of both questions and answers and their mutual relationships by just using an off-the-shelf shallow syntactic parser. We encode structures in Support Vector Machines (SVMs) by means of sequence and tree kernels, which can implicitly represent question and answer pairs in huge feature spaces. Such models together with the latest approach to fast kernel-based learning enabled the training of our rerankers on hundreds of thousands of instances, which previously rendered intractable for kernelized SVMs. The results on two different QA datasets, e.g., Answerbag and Jeopardy! data, show that our models deliver large improvement on passage re-ranking tasks, reducing the error in Recall of BM25 baseline by about 18%. One of the key findings of this work is that, despite its simplicity, shallow syntactic trees allow for learning complex relational structures, which exhibits a steep learning curve with the increase in the training size.

Original languageEnglish
Title of host publicationSIGIR'12 - Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval
Pages741-750
Number of pages10
DOIs
Publication statusPublished - 28 Sep 2012
Event35th Annual ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2012 - Portland, OR, United States
Duration: 12 Aug 201216 Aug 2012

Publication series

NameSIGIR'12 - Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval

Other

Other35th Annual ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2012
CountryUnited States
CityPortland, OR
Period12/8/1216/8/12

    Fingerprint

Keywords

  • kernel methods
  • large-scale learning
  • question answering
  • structural kernels
  • support vector machines

ASJC Scopus subject areas

  • Information Systems

Cite this

Severyn, A., & Moschitti, A. (2012). Structural relationships for large-scale learning of answer re-ranking. In SIGIR'12 - Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 741-750). (SIGIR'12 - Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval). https://doi.org/10.1145/2348283.2348383