Building structures from classifiers for passage reranking

Aliaksei Severyn, Massimo Nicosia, Alessandro Moschitti

Research output: Chapter in Book/Report/Conference proceedingConference contribution

23 Citations (Scopus)

Abstract

This paper shows that learning to rank models can be applied to automatically learn complex patterns, such as relational semantic structures occurring in questions and their answer passages. This is achieved by providing the learning algorithm with a tree representation derived from the syntactic trees of questions and passages connected by relational tags, where the latter are again provided by the means of automatic classifiers, i.e., question and focus classifiers and Named Entity Recognizers. This way effective structural relational patterns are implicitly encoded in the representation and can be automatically utilized by powerful machine learning models such as kernel methods. We conduct an extensive experimental evaluation of our models on well-known benchmarks from the question answer (QA) track of TREC challenges. The comparison with state-of-the-art systems and BM25 show a relative improvement in MAP of more than 14% and 45%, respectively. Further comparison on the task restricted to the answer sentence reranking shows an improvement in MAP of more than 8% over the state of the art.

Original languageEnglish
Title of host publicationInternational Conference on Information and Knowledge Management, Proceedings
Pages969-978
Number of pages10
DOIs
Publication statusPublished - 11 Dec 2013
Event22nd ACM International Conference on Information and Knowledge Management, CIKM 2013 - San Francisco, CA, United States
Duration: 27 Oct 20131 Nov 2013

Other

Other22nd ACM International Conference on Information and Knowledge Management, CIKM 2013
CountryUnited States
CitySan Francisco, CA
Period27/10/131/11/13

Fingerprint

Reranking
Classifier
Evaluation
Learning to rank
Tag
Kernel methods
Learning model
Named entity
Machine learning
Learning algorithm
Benchmark

Keywords

  • Kernel methods
  • Learning to rank
  • Question answering
  • Structural kernels

ASJC Scopus subject areas

  • Business, Management and Accounting(all)
  • Decision Sciences(all)

Cite this

Severyn, A., Nicosia, M., & Moschitti, A. (2013). Building structures from classifiers for passage reranking. In International Conference on Information and Knowledge Management, Proceedings (pp. 969-978) https://doi.org/10.1145/2505515.2505688

Building structures from classifiers for passage reranking. / Severyn, Aliaksei; Nicosia, Massimo; Moschitti, Alessandro.

International Conference on Information and Knowledge Management, Proceedings. 2013. p. 969-978.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Severyn, A, Nicosia, M & Moschitti, A 2013, Building structures from classifiers for passage reranking. in International Conference on Information and Knowledge Management, Proceedings. pp. 969-978, 22nd ACM International Conference on Information and Knowledge Management, CIKM 2013, San Francisco, CA, United States, 27/10/13. https://doi.org/10.1145/2505515.2505688
Severyn A, Nicosia M, Moschitti A. Building structures from classifiers for passage reranking. In International Conference on Information and Knowledge Management, Proceedings. 2013. p. 969-978 https://doi.org/10.1145/2505515.2505688
Severyn, Aliaksei ; Nicosia, Massimo ; Moschitti, Alessandro. / Building structures from classifiers for passage reranking. International Conference on Information and Knowledge Management, Proceedings. 2013. pp. 969-978
@inproceedings{de63528dd9f24fc1853dfd8d18c58275,
title = "Building structures from classifiers for passage reranking",
abstract = "This paper shows that learning to rank models can be applied to automatically learn complex patterns, such as relational semantic structures occurring in questions and their answer passages. This is achieved by providing the learning algorithm with a tree representation derived from the syntactic trees of questions and passages connected by relational tags, where the latter are again provided by the means of automatic classifiers, i.e., question and focus classifiers and Named Entity Recognizers. This way effective structural relational patterns are implicitly encoded in the representation and can be automatically utilized by powerful machine learning models such as kernel methods. We conduct an extensive experimental evaluation of our models on well-known benchmarks from the question answer (QA) track of TREC challenges. The comparison with state-of-the-art systems and BM25 show a relative improvement in MAP of more than 14{\%} and 45{\%}, respectively. Further comparison on the task restricted to the answer sentence reranking shows an improvement in MAP of more than 8{\%} over the state of the art.",
keywords = "Kernel methods, Learning to rank, Question answering, Structural kernels",
author = "Aliaksei Severyn and Massimo Nicosia and Alessandro Moschitti",
year = "2013",
month = "12",
day = "11",
doi = "10.1145/2505515.2505688",
language = "English",
isbn = "9781450322638",
pages = "969--978",
booktitle = "International Conference on Information and Knowledge Management, Proceedings",

}

TY - GEN

T1 - Building structures from classifiers for passage reranking

AU - Severyn, Aliaksei

AU - Nicosia, Massimo

AU - Moschitti, Alessandro

PY - 2013/12/11

Y1 - 2013/12/11

N2 - This paper shows that learning to rank models can be applied to automatically learn complex patterns, such as relational semantic structures occurring in questions and their answer passages. This is achieved by providing the learning algorithm with a tree representation derived from the syntactic trees of questions and passages connected by relational tags, where the latter are again provided by the means of automatic classifiers, i.e., question and focus classifiers and Named Entity Recognizers. This way effective structural relational patterns are implicitly encoded in the representation and can be automatically utilized by powerful machine learning models such as kernel methods. We conduct an extensive experimental evaluation of our models on well-known benchmarks from the question answer (QA) track of TREC challenges. The comparison with state-of-the-art systems and BM25 show a relative improvement in MAP of more than 14% and 45%, respectively. Further comparison on the task restricted to the answer sentence reranking shows an improvement in MAP of more than 8% over the state of the art.

AB - This paper shows that learning to rank models can be applied to automatically learn complex patterns, such as relational semantic structures occurring in questions and their answer passages. This is achieved by providing the learning algorithm with a tree representation derived from the syntactic trees of questions and passages connected by relational tags, where the latter are again provided by the means of automatic classifiers, i.e., question and focus classifiers and Named Entity Recognizers. This way effective structural relational patterns are implicitly encoded in the representation and can be automatically utilized by powerful machine learning models such as kernel methods. We conduct an extensive experimental evaluation of our models on well-known benchmarks from the question answer (QA) track of TREC challenges. The comparison with state-of-the-art systems and BM25 show a relative improvement in MAP of more than 14% and 45%, respectively. Further comparison on the task restricted to the answer sentence reranking shows an improvement in MAP of more than 8% over the state of the art.

KW - Kernel methods

KW - Learning to rank

KW - Question answering

KW - Structural kernels

UR - http://www.scopus.com/inward/record.url?scp=84889594940&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84889594940&partnerID=8YFLogxK

U2 - 10.1145/2505515.2505688

DO - 10.1145/2505515.2505688

M3 - Conference contribution

SN - 9781450322638

SP - 969

EP - 978

BT - International Conference on Information and Knowledge Management, Proceedings

ER -