Hypotheses selection for re-ranking semantic annotations

Marco Dinarelli, Alessandro Moschitti, Giuseppe Riccardi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

Discriminative reranking has been successfully used for several tasks of Natural Language Processing (NLP). Recently it has been applied also to Spoken Language Understanding, imrpoving state-of-the-art for some applications. However, such proposed models can be further improved by considering: (i) a better selection of the initial nbest hypotheses to be re-ranked and (ii) the use of a strategy that decides when the reranking model should be used, i.e. in some cases only the basic approach should be applied. In this paper, we apply a semantic inconsistency metric to select the n-best hypotheses from a large set generated by an SLU basic system. Then we apply a state-of-the-art re-ranker based on the Partial Tree Kernel (PTK), which encodes SLU hypotheses in Support Vector Machines (SVM) with complex structured features. Finally, we apply a decision model based on confidence values to select between the first hypothesis provided by the basic SLU model and the first hypothesis provided by the re-ranker. We show the effectiveness of our approach presenting comparative results obtained by reranking hypotheses generated by two very different models: a simple Stochastic Language Model encoded in Finite State Machines (FSM) and a Conditional Random Field (CRF) model. We evaluate our approach on the French MEDIA corpus and on an Italian corpus acquired in the European Project LUNA. The results show a significant improvement with respect to the current state-of-the-art and previous re-ranking models.

Original languageEnglish
Title of host publication2010 IEEE Workshop on Spoken Language Technology, SLT 2010 - Proceedings
Pages407-411
Number of pages5
DOIs
Publication statusPublished - 1 Dec 2010
Externally publishedYes
Event2010 IEEE Workshop on Spoken Language Technology, SLT 2010 - Berkeley, CA, United States
Duration: 12 Dec 201015 Dec 2010

Other

Other2010 IEEE Workshop on Spoken Language Technology, SLT 2010
CountryUnited States
CityBerkeley, CA
Period12/12/1015/12/10

Fingerprint

Annotation
Ranking
European Project
Inconsistency
Support Vector Machine
Natural Language Processing
Confidence
Language Model
Spoken Language Understanding
Kernel

Keywords

  • Discriminative reranking
  • Kernel methods
  • Spoken language understanding

ASJC Scopus subject areas

  • Language and Linguistics

Cite this

Dinarelli, M., Moschitti, A., & Riccardi, G. (2010). Hypotheses selection for re-ranking semantic annotations. In 2010 IEEE Workshop on Spoken Language Technology, SLT 2010 - Proceedings (pp. 407-411). [5700887] https://doi.org/10.1109/SLT.2010.5700887

Hypotheses selection for re-ranking semantic annotations. / Dinarelli, Marco; Moschitti, Alessandro; Riccardi, Giuseppe.

2010 IEEE Workshop on Spoken Language Technology, SLT 2010 - Proceedings. 2010. p. 407-411 5700887.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Dinarelli, M, Moschitti, A & Riccardi, G 2010, Hypotheses selection for re-ranking semantic annotations. in 2010 IEEE Workshop on Spoken Language Technology, SLT 2010 - Proceedings., 5700887, pp. 407-411, 2010 IEEE Workshop on Spoken Language Technology, SLT 2010, Berkeley, CA, United States, 12/12/10. https://doi.org/10.1109/SLT.2010.5700887
Dinarelli M, Moschitti A, Riccardi G. Hypotheses selection for re-ranking semantic annotations. In 2010 IEEE Workshop on Spoken Language Technology, SLT 2010 - Proceedings. 2010. p. 407-411. 5700887 https://doi.org/10.1109/SLT.2010.5700887
Dinarelli, Marco ; Moschitti, Alessandro ; Riccardi, Giuseppe. / Hypotheses selection for re-ranking semantic annotations. 2010 IEEE Workshop on Spoken Language Technology, SLT 2010 - Proceedings. 2010. pp. 407-411
@inproceedings{3ae89157364640f4b3f96873e391b6a7,
title = "Hypotheses selection for re-ranking semantic annotations",
abstract = "Discriminative reranking has been successfully used for several tasks of Natural Language Processing (NLP). Recently it has been applied also to Spoken Language Understanding, imrpoving state-of-the-art for some applications. However, such proposed models can be further improved by considering: (i) a better selection of the initial nbest hypotheses to be re-ranked and (ii) the use of a strategy that decides when the reranking model should be used, i.e. in some cases only the basic approach should be applied. In this paper, we apply a semantic inconsistency metric to select the n-best hypotheses from a large set generated by an SLU basic system. Then we apply a state-of-the-art re-ranker based on the Partial Tree Kernel (PTK), which encodes SLU hypotheses in Support Vector Machines (SVM) with complex structured features. Finally, we apply a decision model based on confidence values to select between the first hypothesis provided by the basic SLU model and the first hypothesis provided by the re-ranker. We show the effectiveness of our approach presenting comparative results obtained by reranking hypotheses generated by two very different models: a simple Stochastic Language Model encoded in Finite State Machines (FSM) and a Conditional Random Field (CRF) model. We evaluate our approach on the French MEDIA corpus and on an Italian corpus acquired in the European Project LUNA. The results show a significant improvement with respect to the current state-of-the-art and previous re-ranking models.",
keywords = "Discriminative reranking, Kernel methods, Spoken language understanding",
author = "Marco Dinarelli and Alessandro Moschitti and Giuseppe Riccardi",
year = "2010",
month = "12",
day = "1",
doi = "10.1109/SLT.2010.5700887",
language = "English",
isbn = "9781424479030",
pages = "407--411",
booktitle = "2010 IEEE Workshop on Spoken Language Technology, SLT 2010 - Proceedings",

}

TY - GEN

T1 - Hypotheses selection for re-ranking semantic annotations

AU - Dinarelli, Marco

AU - Moschitti, Alessandro

AU - Riccardi, Giuseppe

PY - 2010/12/1

Y1 - 2010/12/1

N2 - Discriminative reranking has been successfully used for several tasks of Natural Language Processing (NLP). Recently it has been applied also to Spoken Language Understanding, imrpoving state-of-the-art for some applications. However, such proposed models can be further improved by considering: (i) a better selection of the initial nbest hypotheses to be re-ranked and (ii) the use of a strategy that decides when the reranking model should be used, i.e. in some cases only the basic approach should be applied. In this paper, we apply a semantic inconsistency metric to select the n-best hypotheses from a large set generated by an SLU basic system. Then we apply a state-of-the-art re-ranker based on the Partial Tree Kernel (PTK), which encodes SLU hypotheses in Support Vector Machines (SVM) with complex structured features. Finally, we apply a decision model based on confidence values to select between the first hypothesis provided by the basic SLU model and the first hypothesis provided by the re-ranker. We show the effectiveness of our approach presenting comparative results obtained by reranking hypotheses generated by two very different models: a simple Stochastic Language Model encoded in Finite State Machines (FSM) and a Conditional Random Field (CRF) model. We evaluate our approach on the French MEDIA corpus and on an Italian corpus acquired in the European Project LUNA. The results show a significant improvement with respect to the current state-of-the-art and previous re-ranking models.

AB - Discriminative reranking has been successfully used for several tasks of Natural Language Processing (NLP). Recently it has been applied also to Spoken Language Understanding, imrpoving state-of-the-art for some applications. However, such proposed models can be further improved by considering: (i) a better selection of the initial nbest hypotheses to be re-ranked and (ii) the use of a strategy that decides when the reranking model should be used, i.e. in some cases only the basic approach should be applied. In this paper, we apply a semantic inconsistency metric to select the n-best hypotheses from a large set generated by an SLU basic system. Then we apply a state-of-the-art re-ranker based on the Partial Tree Kernel (PTK), which encodes SLU hypotheses in Support Vector Machines (SVM) with complex structured features. Finally, we apply a decision model based on confidence values to select between the first hypothesis provided by the basic SLU model and the first hypothesis provided by the re-ranker. We show the effectiveness of our approach presenting comparative results obtained by reranking hypotheses generated by two very different models: a simple Stochastic Language Model encoded in Finite State Machines (FSM) and a Conditional Random Field (CRF) model. We evaluate our approach on the French MEDIA corpus and on an Italian corpus acquired in the European Project LUNA. The results show a significant improvement with respect to the current state-of-the-art and previous re-ranking models.

KW - Discriminative reranking

KW - Kernel methods

KW - Spoken language understanding

UR - http://www.scopus.com/inward/record.url?scp=79951782390&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79951782390&partnerID=8YFLogxK

U2 - 10.1109/SLT.2010.5700887

DO - 10.1109/SLT.2010.5700887

M3 - Conference contribution

SN - 9781424479030

SP - 407

EP - 411

BT - 2010 IEEE Workshop on Spoken Language Technology, SLT 2010 - Proceedings

ER -