Evaluating a dialog language generation system

Comparing the MOUNTAIN system to other NLG approaches

Brian Langner, Stephan Vogel, Alan W. Black

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

This paper describes the MOUNTAIN language generation system, a fully-automatic, data-driven approach to natural language generation aimed at spoken dialog applications. MOUNTAIN uses statistical machine translation techniques and natural corpora to generate human-like language from a structured internal language, such as a representation of the dialog state. We briefly describe the training process for the MOUNTAIN approach, and show results of automatic evaluation in a standard language generation domain: the METEO weather forecasting corpus. Further, we compare output from the MOUNTAIN system to several other NLG systems in the same domain, using both automatic and human-based evaluation metrics; our results show our approach is comparable in quality to other advanced approaches. Finally, we discuss potential extensions, improvements, and other planned tests.

Original languageEnglish
Title of host publicationProceedings of the 11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010
Pages1109-1112
Number of pages4
Publication statusPublished - 1 Dec 2010
Externally publishedYes
Event11th Annual Conference of the International Speech Communication Association: Spoken Language Processing for All, INTERSPEECH 2010 - Makuhari, Chiba, Japan
Duration: 26 Sep 201030 Sep 2010

Other

Other11th Annual Conference of the International Speech Communication Association: Spoken Language Processing for All, INTERSPEECH 2010
CountryJapan
CityMakuhari, Chiba
Period26/9/1030/9/10

Fingerprint

Language
Weather
Language Generation
Evaluation

Keywords

  • Evaluation
  • Natural language generation
  • Spoken dialog
  • Translation-based generation
  • Weather forecasts

ASJC Scopus subject areas

  • Language and Linguistics
  • Speech and Hearing

Cite this

Langner, B., Vogel, S., & Black, A. W. (2010). Evaluating a dialog language generation system: Comparing the MOUNTAIN system to other NLG approaches. In Proceedings of the 11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010 (pp. 1109-1112)

Evaluating a dialog language generation system : Comparing the MOUNTAIN system to other NLG approaches. / Langner, Brian; Vogel, Stephan; Black, Alan W.

Proceedings of the 11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010. 2010. p. 1109-1112.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Langner, B, Vogel, S & Black, AW 2010, Evaluating a dialog language generation system: Comparing the MOUNTAIN system to other NLG approaches. in Proceedings of the 11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010. pp. 1109-1112, 11th Annual Conference of the International Speech Communication Association: Spoken Language Processing for All, INTERSPEECH 2010, Makuhari, Chiba, Japan, 26/9/10.
Langner B, Vogel S, Black AW. Evaluating a dialog language generation system: Comparing the MOUNTAIN system to other NLG approaches. In Proceedings of the 11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010. 2010. p. 1109-1112
Langner, Brian ; Vogel, Stephan ; Black, Alan W. / Evaluating a dialog language generation system : Comparing the MOUNTAIN system to other NLG approaches. Proceedings of the 11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010. 2010. pp. 1109-1112
@inproceedings{63e415a1794947ce8dfffc86a6c4ab1f,
title = "Evaluating a dialog language generation system: Comparing the MOUNTAIN system to other NLG approaches",
abstract = "This paper describes the MOUNTAIN language generation system, a fully-automatic, data-driven approach to natural language generation aimed at spoken dialog applications. MOUNTAIN uses statistical machine translation techniques and natural corpora to generate human-like language from a structured internal language, such as a representation of the dialog state. We briefly describe the training process for the MOUNTAIN approach, and show results of automatic evaluation in a standard language generation domain: the METEO weather forecasting corpus. Further, we compare output from the MOUNTAIN system to several other NLG systems in the same domain, using both automatic and human-based evaluation metrics; our results show our approach is comparable in quality to other advanced approaches. Finally, we discuss potential extensions, improvements, and other planned tests.",
keywords = "Evaluation, Natural language generation, Spoken dialog, Translation-based generation, Weather forecasts",
author = "Brian Langner and Stephan Vogel and Black, {Alan W.}",
year = "2010",
month = "12",
day = "1",
language = "English",
pages = "1109--1112",
booktitle = "Proceedings of the 11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010",

}

TY - GEN

T1 - Evaluating a dialog language generation system

T2 - Comparing the MOUNTAIN system to other NLG approaches

AU - Langner, Brian

AU - Vogel, Stephan

AU - Black, Alan W.

PY - 2010/12/1

Y1 - 2010/12/1

N2 - This paper describes the MOUNTAIN language generation system, a fully-automatic, data-driven approach to natural language generation aimed at spoken dialog applications. MOUNTAIN uses statistical machine translation techniques and natural corpora to generate human-like language from a structured internal language, such as a representation of the dialog state. We briefly describe the training process for the MOUNTAIN approach, and show results of automatic evaluation in a standard language generation domain: the METEO weather forecasting corpus. Further, we compare output from the MOUNTAIN system to several other NLG systems in the same domain, using both automatic and human-based evaluation metrics; our results show our approach is comparable in quality to other advanced approaches. Finally, we discuss potential extensions, improvements, and other planned tests.

AB - This paper describes the MOUNTAIN language generation system, a fully-automatic, data-driven approach to natural language generation aimed at spoken dialog applications. MOUNTAIN uses statistical machine translation techniques and natural corpora to generate human-like language from a structured internal language, such as a representation of the dialog state. We briefly describe the training process for the MOUNTAIN approach, and show results of automatic evaluation in a standard language generation domain: the METEO weather forecasting corpus. Further, we compare output from the MOUNTAIN system to several other NLG systems in the same domain, using both automatic and human-based evaluation metrics; our results show our approach is comparable in quality to other advanced approaches. Finally, we discuss potential extensions, improvements, and other planned tests.

KW - Evaluation

KW - Natural language generation

KW - Spoken dialog

KW - Translation-based generation

KW - Weather forecasts

UR - http://www.scopus.com/inward/record.url?scp=79959828072&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79959828072&partnerID=8YFLogxK

M3 - Conference contribution

SP - 1109

EP - 1112

BT - Proceedings of the 11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010

ER -