Evaluating information extraction

Andrea Esuli, Fabrizio Sebastiani

Research output: Chapter in Book/Report/Conference proceedingConference contribution

7 Citations (Scopus)

Abstract

The issue of how to experimentally evaluate information extraction (IE) systems has received hardly any satisfactory solution in the literature. In this paper we propose a novel evaluation model for IE and argue that, among others, it allows (i) a correct appreciation of the degree of overlap between predicted and true segments, and (ii) a fair evaluation of the ability of a system to correctly identify segment boundaries. We describe the properties of this models, also by presenting the result of a re-evaluation of the results of the CoNLL'03 and CoNLL'02 Shared Tasks on Named Entity Extraction.

Original languageEnglish
Title of host publicationMultilingual and Multimodal Information Access Evaluation - International Conference of the Cross-Language Evaluation Forum, CLEF 2010, Proceedings
Pages100-111
Number of pages12
DOIs
Publication statusPublished - 8 Nov 2010
EventInternational Conference of the Cross-Language Evaluation Forum, CLEF 2010 - Padua, Italy
Duration: 20 Sep 201023 Sep 2010

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume6360 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

OtherInternational Conference of the Cross-Language Evaluation Forum, CLEF 2010
CountryItaly
CityPadua
Period20/9/1023/9/10

    Fingerprint

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Esuli, A., & Sebastiani, F. (2010). Evaluating information extraction. In Multilingual and Multimodal Information Access Evaluation - International Conference of the Cross-Language Evaluation Forum, CLEF 2010, Proceedings (pp. 100-111). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 6360 LNCS). https://doi.org/10.1007/978-3-642-15998-5_12