Do automatic annotation techniques have any impact on supervised complex question answering?

Yllias Chali, Sadid A. Hasan, Shafiq Rayhan Joty

Research output: Chapter in Book/Report/Conference proceedingConference contribution

8 Citations (Scopus)

Abstract

In this paper, we analyze the impact of different automatic annotation methods on the performance of supervised approaches to the complex question answering problem (defined in the DUC-2007 main task). Huge amount of annotated or labeled data is a prerequisite for supervised training. The task of labeling can be accomplished either by humans or by computer programs. When humans are employed, the whole process becomes time consuming and expensive. So, in order to produce a large set of labeled data we prefer the automatic annotation strategy. We apply five different automatic annotation techniques to produce labeled data using ROUGE similarity measure, Basic Element (BE) overlap, syntactic similarity measure, semantic similarity measure, and Extended String Subsequence Kernel (ESSK). The representative supervised methods we use are Support Vector Machines (SVM), Conditional Random Fields (CRF), Hidden Markov Models (HMM), and Maximum Entropy (Max-Ent). Evaluation results are presented to show the impact.

Original languageEnglish
Title of host publicationACL-IJCNLP 2009 - Joint Conf. of the 47th Annual Meeting of the Association for Computational Linguistics and 4th Int. Joint Conf. on Natural Language Processing of the AFNLP, Proceedings of the Conf.
Pages329-332
Number of pages4
Publication statusPublished - 1 Dec 2009
Externally publishedYes
EventJoint Conference of the 47th Annual Meeting of the Association for Computational Linguistics and 4th International Joint Conference on Natural Language Processing of the AFNLP, ACL-IJCNLP 2009 - Suntec, Singapore
Duration: 2 Aug 20097 Aug 2009

Other

OtherJoint Conference of the 47th Annual Meeting of the Association for Computational Linguistics and 4th International Joint Conference on Natural Language Processing of the AFNLP, ACL-IJCNLP 2009
CountrySingapore
CitySuntec
Period2/8/097/8/09

Fingerprint

data processing program
entropy
semantics
evaluation
performance
Question Answering
Annotation
Complex Question
time
Overlap
Hidden Markov Model
Maximum Entropy
Labeling
Syntax
Kernel
Evaluation
Semantic Similarity
Support Vector Machine
Strings

ASJC Scopus subject areas

  • Language and Linguistics
  • Linguistics and Language

Cite this

Chali, Y., Hasan, S. A., & Rayhan Joty, S. (2009). Do automatic annotation techniques have any impact on supervised complex question answering? In ACL-IJCNLP 2009 - Joint Conf. of the 47th Annual Meeting of the Association for Computational Linguistics and 4th Int. Joint Conf. on Natural Language Processing of the AFNLP, Proceedings of the Conf. (pp. 329-332)

Do automatic annotation techniques have any impact on supervised complex question answering? / Chali, Yllias; Hasan, Sadid A.; Rayhan Joty, Shafiq.

ACL-IJCNLP 2009 - Joint Conf. of the 47th Annual Meeting of the Association for Computational Linguistics and 4th Int. Joint Conf. on Natural Language Processing of the AFNLP, Proceedings of the Conf.. 2009. p. 329-332.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Chali, Y, Hasan, SA & Rayhan Joty, S 2009, Do automatic annotation techniques have any impact on supervised complex question answering? in ACL-IJCNLP 2009 - Joint Conf. of the 47th Annual Meeting of the Association for Computational Linguistics and 4th Int. Joint Conf. on Natural Language Processing of the AFNLP, Proceedings of the Conf.. pp. 329-332, Joint Conference of the 47th Annual Meeting of the Association for Computational Linguistics and 4th International Joint Conference on Natural Language Processing of the AFNLP, ACL-IJCNLP 2009, Suntec, Singapore, 2/8/09.
Chali Y, Hasan SA, Rayhan Joty S. Do automatic annotation techniques have any impact on supervised complex question answering? In ACL-IJCNLP 2009 - Joint Conf. of the 47th Annual Meeting of the Association for Computational Linguistics and 4th Int. Joint Conf. on Natural Language Processing of the AFNLP, Proceedings of the Conf.. 2009. p. 329-332
Chali, Yllias ; Hasan, Sadid A. ; Rayhan Joty, Shafiq. / Do automatic annotation techniques have any impact on supervised complex question answering?. ACL-IJCNLP 2009 - Joint Conf. of the 47th Annual Meeting of the Association for Computational Linguistics and 4th Int. Joint Conf. on Natural Language Processing of the AFNLP, Proceedings of the Conf.. 2009. pp. 329-332
@inproceedings{903f6480be7f40239852b53d1cd7a6fd,
title = "Do automatic annotation techniques have any impact on supervised complex question answering?",
abstract = "In this paper, we analyze the impact of different automatic annotation methods on the performance of supervised approaches to the complex question answering problem (defined in the DUC-2007 main task). Huge amount of annotated or labeled data is a prerequisite for supervised training. The task of labeling can be accomplished either by humans or by computer programs. When humans are employed, the whole process becomes time consuming and expensive. So, in order to produce a large set of labeled data we prefer the automatic annotation strategy. We apply five different automatic annotation techniques to produce labeled data using ROUGE similarity measure, Basic Element (BE) overlap, syntactic similarity measure, semantic similarity measure, and Extended String Subsequence Kernel (ESSK). The representative supervised methods we use are Support Vector Machines (SVM), Conditional Random Fields (CRF), Hidden Markov Models (HMM), and Maximum Entropy (Max-Ent). Evaluation results are presented to show the impact.",
author = "Yllias Chali and Hasan, {Sadid A.} and {Rayhan Joty}, Shafiq",
year = "2009",
month = "12",
day = "1",
language = "English",
isbn = "9781617382581",
pages = "329--332",
booktitle = "ACL-IJCNLP 2009 - Joint Conf. of the 47th Annual Meeting of the Association for Computational Linguistics and 4th Int. Joint Conf. on Natural Language Processing of the AFNLP, Proceedings of the Conf.",

}

TY - GEN

T1 - Do automatic annotation techniques have any impact on supervised complex question answering?

AU - Chali, Yllias

AU - Hasan, Sadid A.

AU - Rayhan Joty, Shafiq

PY - 2009/12/1

Y1 - 2009/12/1

N2 - In this paper, we analyze the impact of different automatic annotation methods on the performance of supervised approaches to the complex question answering problem (defined in the DUC-2007 main task). Huge amount of annotated or labeled data is a prerequisite for supervised training. The task of labeling can be accomplished either by humans or by computer programs. When humans are employed, the whole process becomes time consuming and expensive. So, in order to produce a large set of labeled data we prefer the automatic annotation strategy. We apply five different automatic annotation techniques to produce labeled data using ROUGE similarity measure, Basic Element (BE) overlap, syntactic similarity measure, semantic similarity measure, and Extended String Subsequence Kernel (ESSK). The representative supervised methods we use are Support Vector Machines (SVM), Conditional Random Fields (CRF), Hidden Markov Models (HMM), and Maximum Entropy (Max-Ent). Evaluation results are presented to show the impact.

AB - In this paper, we analyze the impact of different automatic annotation methods on the performance of supervised approaches to the complex question answering problem (defined in the DUC-2007 main task). Huge amount of annotated or labeled data is a prerequisite for supervised training. The task of labeling can be accomplished either by humans or by computer programs. When humans are employed, the whole process becomes time consuming and expensive. So, in order to produce a large set of labeled data we prefer the automatic annotation strategy. We apply five different automatic annotation techniques to produce labeled data using ROUGE similarity measure, Basic Element (BE) overlap, syntactic similarity measure, semantic similarity measure, and Extended String Subsequence Kernel (ESSK). The representative supervised methods we use are Support Vector Machines (SVM), Conditional Random Fields (CRF), Hidden Markov Models (HMM), and Maximum Entropy (Max-Ent). Evaluation results are presented to show the impact.

UR - http://www.scopus.com/inward/record.url?scp=79960602319&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79960602319&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:79960602319

SN - 9781617382581

SP - 329

EP - 332

BT - ACL-IJCNLP 2009 - Joint Conf. of the 47th Annual Meeting of the Association for Computational Linguistics and 4th Int. Joint Conf. on Natural Language Processing of the AFNLP, Proceedings of the Conf.

ER -