Evaluating information extraction

Andrea Esuli, Fabrizio Sebastiani

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

The issue of how to experimentally evaluate information extraction (IE) systems has received hardly any satisfactory solution in the literature. In this paper we propose a novel evaluation model for IE and argue that, among others, it allows (i) a correct appreciation of the degree of overlap between predicted and true segments, and (ii) a fair evaluation of the ability of a system to correctly identify segment boundaries. We describe the properties of this models, also by presenting the result of a re-evaluation of the results of the CoNLL'03 and CoNLL'02 Shared Tasks on Named Entity Extraction.

Original languageEnglish
Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Pages100-111
Number of pages12
Volume6360 LNCS
DOIs
Publication statusPublished - 2010
Externally publishedYes
EventInternational Conference of the Cross-Language Evaluation Forum, CLEF 2010 - Padua, Italy
Duration: 20 Sep 201023 Sep 2010

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume6360 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

OtherInternational Conference of the Cross-Language Evaluation Forum, CLEF 2010
CountryItaly
CityPadua
Period20/9/1023/9/10

Fingerprint

Information Extraction
Evaluation Model
Evaluation
Overlap
Evaluate
Model

ASJC Scopus subject areas

  • Computer Science(all)
  • Theoretical Computer Science

Cite this

Esuli, A., & Sebastiani, F. (2010). Evaluating information extraction. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 6360 LNCS, pp. 100-111). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 6360 LNCS). https://doi.org/10.1007/978-3-642-15998-5_12

Evaluating information extraction. / Esuli, Andrea; Sebastiani, Fabrizio.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 6360 LNCS 2010. p. 100-111 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 6360 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Esuli, A & Sebastiani, F 2010, Evaluating information extraction. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). vol. 6360 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 6360 LNCS, pp. 100-111, International Conference of the Cross-Language Evaluation Forum, CLEF 2010, Padua, Italy, 20/9/10. https://doi.org/10.1007/978-3-642-15998-5_12
Esuli A, Sebastiani F. Evaluating information extraction. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 6360 LNCS. 2010. p. 100-111. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-642-15998-5_12
Esuli, Andrea ; Sebastiani, Fabrizio. / Evaluating information extraction. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 6360 LNCS 2010. pp. 100-111 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{32a71c0feab84e7aa8308a43cd48c19e,
title = "Evaluating information extraction",
abstract = "The issue of how to experimentally evaluate information extraction (IE) systems has received hardly any satisfactory solution in the literature. In this paper we propose a novel evaluation model for IE and argue that, among others, it allows (i) a correct appreciation of the degree of overlap between predicted and true segments, and (ii) a fair evaluation of the ability of a system to correctly identify segment boundaries. We describe the properties of this models, also by presenting the result of a re-evaluation of the results of the CoNLL'03 and CoNLL'02 Shared Tasks on Named Entity Extraction.",
author = "Andrea Esuli and Fabrizio Sebastiani",
year = "2010",
doi = "10.1007/978-3-642-15998-5_12",
language = "English",
isbn = "3642159974",
volume = "6360 LNCS",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
pages = "100--111",
booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

}

TY - GEN

T1 - Evaluating information extraction

AU - Esuli, Andrea

AU - Sebastiani, Fabrizio

PY - 2010

Y1 - 2010

N2 - The issue of how to experimentally evaluate information extraction (IE) systems has received hardly any satisfactory solution in the literature. In this paper we propose a novel evaluation model for IE and argue that, among others, it allows (i) a correct appreciation of the degree of overlap between predicted and true segments, and (ii) a fair evaluation of the ability of a system to correctly identify segment boundaries. We describe the properties of this models, also by presenting the result of a re-evaluation of the results of the CoNLL'03 and CoNLL'02 Shared Tasks on Named Entity Extraction.

AB - The issue of how to experimentally evaluate information extraction (IE) systems has received hardly any satisfactory solution in the literature. In this paper we propose a novel evaluation model for IE and argue that, among others, it allows (i) a correct appreciation of the degree of overlap between predicted and true segments, and (ii) a fair evaluation of the ability of a system to correctly identify segment boundaries. We describe the properties of this models, also by presenting the result of a re-evaluation of the results of the CoNLL'03 and CoNLL'02 Shared Tasks on Named Entity Extraction.

UR - http://www.scopus.com/inward/record.url?scp=78049384057&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=78049384057&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-15998-5_12

DO - 10.1007/978-3-642-15998-5_12

M3 - Conference contribution

SN - 3642159974

SN - 9783642159978

VL - 6360 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 100

EP - 111

BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

ER -