Discriminative reranking for spoken language understanding

Marco Dinarelli, Alessandro Moschitti, Giuseppe Riccardi

Research output: Contribution to journalArticle

18 Citations (Scopus)

Abstract

Spoken language understanding (SLU) is concerned with the extraction of meaning structures from spoken utterances. Recent computational approaches to SLU, e.g., conditional random fields (CRFs), optimize local models by encoding several features, mainly based on simple n-grams. In contrast, recent works have shown that the accuracy of CRF can be significantly improved by modeling long-distance dependency features. In this paper, we propose novel approaches to encode all possible dependencies between features and most importantly among parts of the meaning structure, e.g., concepts and their combination. We rerank hypotheses generated by local models, e.g., stochastic finite state transducers (SFSTs) or CRF, with a global model. The latter encodes a very large number of dependencies (in the form of trees or sequences) by applying kernel methods to the space of all meaning (sub) structures. We performed comparative experiments between SFST, CRF, support vector machines (SVMs), and our proposed discriminative reranking models (DRMs) on representative conversational speech corpora in three different languages: the ATIS (English), the MEDIA (French), and the LUNA (Italian) corpora. These corpora have been collected within three different domain applications of increasing complexity: informational, transactional, and problem-solving tasks, respectively. The results show that our DRMs consistently outperform the state-of-the-art models based on CRF.

Original languageEnglish
Article number5955081
Pages (from-to)526-539
Number of pages14
JournalIEEE Transactions on Audio, Speech and Language Processing
Volume20
Issue number2
DOIs
Publication statusPublished - 1 Jan 2012
Externally publishedYes

Fingerprint

Transducers
transducers
Stochastic models
problem solving
Support vector machines
coding
Experiments

Keywords

  • Kernel methods
  • machine learning
  • natural language processing (NLP)
  • spoken language understanding (SLU)
  • stochastic language models
  • support vector machines (SVMs)

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Acoustics and Ultrasonics

Cite this

Discriminative reranking for spoken language understanding. / Dinarelli, Marco; Moschitti, Alessandro; Riccardi, Giuseppe.

In: IEEE Transactions on Audio, Speech and Language Processing, Vol. 20, No. 2, 5955081, 01.01.2012, p. 526-539.

Research output: Contribution to journalArticle

@article{86833b6f255d47a1a7fb8808bc717396,
title = "Discriminative reranking for spoken language understanding",
abstract = "Spoken language understanding (SLU) is concerned with the extraction of meaning structures from spoken utterances. Recent computational approaches to SLU, e.g., conditional random fields (CRFs), optimize local models by encoding several features, mainly based on simple n-grams. In contrast, recent works have shown that the accuracy of CRF can be significantly improved by modeling long-distance dependency features. In this paper, we propose novel approaches to encode all possible dependencies between features and most importantly among parts of the meaning structure, e.g., concepts and their combination. We rerank hypotheses generated by local models, e.g., stochastic finite state transducers (SFSTs) or CRF, with a global model. The latter encodes a very large number of dependencies (in the form of trees or sequences) by applying kernel methods to the space of all meaning (sub) structures. We performed comparative experiments between SFST, CRF, support vector machines (SVMs), and our proposed discriminative reranking models (DRMs) on representative conversational speech corpora in three different languages: the ATIS (English), the MEDIA (French), and the LUNA (Italian) corpora. These corpora have been collected within three different domain applications of increasing complexity: informational, transactional, and problem-solving tasks, respectively. The results show that our DRMs consistently outperform the state-of-the-art models based on CRF.",
keywords = "Kernel methods, machine learning, natural language processing (NLP), spoken language understanding (SLU), stochastic language models, support vector machines (SVMs)",
author = "Marco Dinarelli and Alessandro Moschitti and Giuseppe Riccardi",
year = "2012",
month = "1",
day = "1",
doi = "10.1109/TASL.2011.2162322",
language = "English",
volume = "20",
pages = "526--539",
journal = "IEEE Transactions on Audio, Speech and Language Processing",
issn = "1558-7916",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "2",

}

TY - JOUR

T1 - Discriminative reranking for spoken language understanding

AU - Dinarelli, Marco

AU - Moschitti, Alessandro

AU - Riccardi, Giuseppe

PY - 2012/1/1

Y1 - 2012/1/1

N2 - Spoken language understanding (SLU) is concerned with the extraction of meaning structures from spoken utterances. Recent computational approaches to SLU, e.g., conditional random fields (CRFs), optimize local models by encoding several features, mainly based on simple n-grams. In contrast, recent works have shown that the accuracy of CRF can be significantly improved by modeling long-distance dependency features. In this paper, we propose novel approaches to encode all possible dependencies between features and most importantly among parts of the meaning structure, e.g., concepts and their combination. We rerank hypotheses generated by local models, e.g., stochastic finite state transducers (SFSTs) or CRF, with a global model. The latter encodes a very large number of dependencies (in the form of trees or sequences) by applying kernel methods to the space of all meaning (sub) structures. We performed comparative experiments between SFST, CRF, support vector machines (SVMs), and our proposed discriminative reranking models (DRMs) on representative conversational speech corpora in three different languages: the ATIS (English), the MEDIA (French), and the LUNA (Italian) corpora. These corpora have been collected within three different domain applications of increasing complexity: informational, transactional, and problem-solving tasks, respectively. The results show that our DRMs consistently outperform the state-of-the-art models based on CRF.

AB - Spoken language understanding (SLU) is concerned with the extraction of meaning structures from spoken utterances. Recent computational approaches to SLU, e.g., conditional random fields (CRFs), optimize local models by encoding several features, mainly based on simple n-grams. In contrast, recent works have shown that the accuracy of CRF can be significantly improved by modeling long-distance dependency features. In this paper, we propose novel approaches to encode all possible dependencies between features and most importantly among parts of the meaning structure, e.g., concepts and their combination. We rerank hypotheses generated by local models, e.g., stochastic finite state transducers (SFSTs) or CRF, with a global model. The latter encodes a very large number of dependencies (in the form of trees or sequences) by applying kernel methods to the space of all meaning (sub) structures. We performed comparative experiments between SFST, CRF, support vector machines (SVMs), and our proposed discriminative reranking models (DRMs) on representative conversational speech corpora in three different languages: the ATIS (English), the MEDIA (French), and the LUNA (Italian) corpora. These corpora have been collected within three different domain applications of increasing complexity: informational, transactional, and problem-solving tasks, respectively. The results show that our DRMs consistently outperform the state-of-the-art models based on CRF.

KW - Kernel methods

KW - machine learning

KW - natural language processing (NLP)

KW - spoken language understanding (SLU)

KW - stochastic language models

KW - support vector machines (SVMs)

UR - http://www.scopus.com/inward/record.url?scp=83755174034&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=83755174034&partnerID=8YFLogxK

U2 - 10.1109/TASL.2011.2162322

DO - 10.1109/TASL.2011.2162322

M3 - Article

VL - 20

SP - 526

EP - 539

JO - IEEE Transactions on Audio, Speech and Language Processing

JF - IEEE Transactions on Audio, Speech and Language Processing

SN - 1558-7916

IS - 2

M1 - 5955081

ER -