Linguistic kernels for answer re-ranking in question answering systems

Alessandro Moschitti, Silvia Quarteroni

Research output: Contribution to journalArticle

27 Citations (Scopus)

Abstract

Answer selection is the most complex phase of a question answering (QA) system. To solve this task, typical approaches use unsupervised methods such as computing the similarity between query and answer, optionally exploiting advanced syntactic, semantic or logic representations. In this paper, we study supervised discriminative models that learn to select (rank) answers using examples of question and answer pairs. The pair representation is implicitly provided by kernel combinations applied to each of its members. To reduce the burden of large amounts of manual annotation, we represent question and answer pairs by means of powerful generalization methods, exploiting the application of structural kernels to syntactic/semantic structures. We experiment with support vector machines and string kernels, syntactic and shallow semantic tree kernels applied to part-of-speech tag sequences, syntactic parse trees and predicate argument structures on two datasets which we have compiled and made available. Our results on classification of correct and incorrect pairs show that our best model improves the bag-of-words model by 63% on a TREC dataset. Moreover, such a binary classifier, used as a re-ranker, improves the mean reciprocal rank of our baseline QA system by 13%. These findings demonstrate that our method automatically selects an appropriate representation of question-answer relations.

Original languageEnglish
Pages (from-to)825-842
Number of pages18
JournalInformation Processing and Management
Volume47
Issue number6
DOIs
Publication statusPublished - 1 Nov 2011
Externally publishedYes

Fingerprint

Syntactics
Linguistics
ranking
semantics
linguistics
Semantics
Support vector machines
Classifiers
experiment
Kernel
Reranking
Question answering
Experiments

Keywords

  • Information Retrieval
  • Kernel methods
  • Predicate argument structures
  • Question answering

ASJC Scopus subject areas

  • Media Technology
  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences
  • Management Science and Operations Research

Cite this

Linguistic kernels for answer re-ranking in question answering systems. / Moschitti, Alessandro; Quarteroni, Silvia.

In: Information Processing and Management, Vol. 47, No. 6, 01.11.2011, p. 825-842.

Research output: Contribution to journalArticle

@article{4b6588e2533b4e648ffef98f31b7d830,
title = "Linguistic kernels for answer re-ranking in question answering systems",
abstract = "Answer selection is the most complex phase of a question answering (QA) system. To solve this task, typical approaches use unsupervised methods such as computing the similarity between query and answer, optionally exploiting advanced syntactic, semantic or logic representations. In this paper, we study supervised discriminative models that learn to select (rank) answers using examples of question and answer pairs. The pair representation is implicitly provided by kernel combinations applied to each of its members. To reduce the burden of large amounts of manual annotation, we represent question and answer pairs by means of powerful generalization methods, exploiting the application of structural kernels to syntactic/semantic structures. We experiment with support vector machines and string kernels, syntactic and shallow semantic tree kernels applied to part-of-speech tag sequences, syntactic parse trees and predicate argument structures on two datasets which we have compiled and made available. Our results on classification of correct and incorrect pairs show that our best model improves the bag-of-words model by 63{\%} on a TREC dataset. Moreover, such a binary classifier, used as a re-ranker, improves the mean reciprocal rank of our baseline QA system by 13{\%}. These findings demonstrate that our method automatically selects an appropriate representation of question-answer relations.",
keywords = "Information Retrieval, Kernel methods, Predicate argument structures, Question answering",
author = "Alessandro Moschitti and Silvia Quarteroni",
year = "2011",
month = "11",
day = "1",
doi = "10.1016/j.ipm.2010.06.002",
language = "English",
volume = "47",
pages = "825--842",
journal = "Information Processing and Management",
issn = "0306-4573",
publisher = "Elsevier Limited",
number = "6",

}

TY - JOUR

T1 - Linguistic kernels for answer re-ranking in question answering systems

AU - Moschitti, Alessandro

AU - Quarteroni, Silvia

PY - 2011/11/1

Y1 - 2011/11/1

N2 - Answer selection is the most complex phase of a question answering (QA) system. To solve this task, typical approaches use unsupervised methods such as computing the similarity between query and answer, optionally exploiting advanced syntactic, semantic or logic representations. In this paper, we study supervised discriminative models that learn to select (rank) answers using examples of question and answer pairs. The pair representation is implicitly provided by kernel combinations applied to each of its members. To reduce the burden of large amounts of manual annotation, we represent question and answer pairs by means of powerful generalization methods, exploiting the application of structural kernels to syntactic/semantic structures. We experiment with support vector machines and string kernels, syntactic and shallow semantic tree kernels applied to part-of-speech tag sequences, syntactic parse trees and predicate argument structures on two datasets which we have compiled and made available. Our results on classification of correct and incorrect pairs show that our best model improves the bag-of-words model by 63% on a TREC dataset. Moreover, such a binary classifier, used as a re-ranker, improves the mean reciprocal rank of our baseline QA system by 13%. These findings demonstrate that our method automatically selects an appropriate representation of question-answer relations.

AB - Answer selection is the most complex phase of a question answering (QA) system. To solve this task, typical approaches use unsupervised methods such as computing the similarity between query and answer, optionally exploiting advanced syntactic, semantic or logic representations. In this paper, we study supervised discriminative models that learn to select (rank) answers using examples of question and answer pairs. The pair representation is implicitly provided by kernel combinations applied to each of its members. To reduce the burden of large amounts of manual annotation, we represent question and answer pairs by means of powerful generalization methods, exploiting the application of structural kernels to syntactic/semantic structures. We experiment with support vector machines and string kernels, syntactic and shallow semantic tree kernels applied to part-of-speech tag sequences, syntactic parse trees and predicate argument structures on two datasets which we have compiled and made available. Our results on classification of correct and incorrect pairs show that our best model improves the bag-of-words model by 63% on a TREC dataset. Moreover, such a binary classifier, used as a re-ranker, improves the mean reciprocal rank of our baseline QA system by 13%. These findings demonstrate that our method automatically selects an appropriate representation of question-answer relations.

KW - Information Retrieval

KW - Kernel methods

KW - Predicate argument structures

KW - Question answering

UR - http://www.scopus.com/inward/record.url?scp=80052263476&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=80052263476&partnerID=8YFLogxK

U2 - 10.1016/j.ipm.2010.06.002

DO - 10.1016/j.ipm.2010.06.002

M3 - Article

VL - 47

SP - 825

EP - 842

JO - Information Processing and Management

JF - Information Processing and Management

SN - 0306-4573

IS - 6

ER -