CheckThat! at CLEF 2019

Automatic identification and verification of claims

Tamer Elsayed, Preslav Nakov, Alberto Barron, Maram Hasanain, Reem Suwaileh, Giovanni Martino, Pepa Atanasova

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

We introduce the second edition of the CheckThat! Lab, part of the 2019 Cross-Language Evaluation Forum (CLEF). CheckThat! proposes two complementary tasks. Task 1: predict which claims in a political debate should be prioritized for fact-checking. Task 2: rank Web-retrieved pages against a check-worthy claim based on their usefulness for fact-checking, extract useful passages from those pages, and then use them all to decide whether the claim is factually true or false. Checkthat! provides a full evaluation framework, consisting of data in English (derived from fact-checking sources) and Arabic (gathered and annotated from scratch) and evaluation based on mean average precision (MAP) for ranking and F 1 for classification tasks.

Original languageEnglish
Title of host publicationAdvances in Information Retrieval - 41st European Conference on IR Research, ECIR 2019, Proceedings
EditorsBenno Stein, Philipp Mayr, Leif Azzopardi, Norbert Fuhr, Claudia Hauff, Djoerd Hiemstra
PublisherSpringer Verlag
Pages309-315
Number of pages7
ISBN (Print)9783030157180
DOIs
Publication statusPublished - 1 Jan 2019
Event41st European Conference on Information Retrieval, ECIR 2019 - Cologne, Germany
Duration: 14 Apr 201918 Apr 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11438 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference41st European Conference on Information Retrieval, ECIR 2019
CountryGermany
CityCologne
Period14/4/1918/4/19

Fingerprint

Websites
Evaluation
Ranking
Predict
Language
False
Framework

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Elsayed, T., Nakov, P., Barron, A., Hasanain, M., Suwaileh, R., Martino, G., & Atanasova, P. (2019). CheckThat! at CLEF 2019: Automatic identification and verification of claims. In B. Stein, P. Mayr, L. Azzopardi, N. Fuhr, C. Hauff, & D. Hiemstra (Eds.), Advances in Information Retrieval - 41st European Conference on IR Research, ECIR 2019, Proceedings (pp. 309-315). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11438 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-15719-7_41

CheckThat! at CLEF 2019 : Automatic identification and verification of claims. / Elsayed, Tamer; Nakov, Preslav; Barron, Alberto; Hasanain, Maram; Suwaileh, Reem; Martino, Giovanni; Atanasova, Pepa.

Advances in Information Retrieval - 41st European Conference on IR Research, ECIR 2019, Proceedings. ed. / Benno Stein; Philipp Mayr; Leif Azzopardi; Norbert Fuhr; Claudia Hauff; Djoerd Hiemstra. Springer Verlag, 2019. p. 309-315 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11438 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Elsayed, T, Nakov, P, Barron, A, Hasanain, M, Suwaileh, R, Martino, G & Atanasova, P 2019, CheckThat! at CLEF 2019: Automatic identification and verification of claims. in B Stein, P Mayr, L Azzopardi, N Fuhr, C Hauff & D Hiemstra (eds), Advances in Information Retrieval - 41st European Conference on IR Research, ECIR 2019, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11438 LNCS, Springer Verlag, pp. 309-315, 41st European Conference on Information Retrieval, ECIR 2019, Cologne, Germany, 14/4/19. https://doi.org/10.1007/978-3-030-15719-7_41
Elsayed T, Nakov P, Barron A, Hasanain M, Suwaileh R, Martino G et al. CheckThat! at CLEF 2019: Automatic identification and verification of claims. In Stein B, Mayr P, Azzopardi L, Fuhr N, Hauff C, Hiemstra D, editors, Advances in Information Retrieval - 41st European Conference on IR Research, ECIR 2019, Proceedings. Springer Verlag. 2019. p. 309-315. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-030-15719-7_41
Elsayed, Tamer ; Nakov, Preslav ; Barron, Alberto ; Hasanain, Maram ; Suwaileh, Reem ; Martino, Giovanni ; Atanasova, Pepa. / CheckThat! at CLEF 2019 : Automatic identification and verification of claims. Advances in Information Retrieval - 41st European Conference on IR Research, ECIR 2019, Proceedings. editor / Benno Stein ; Philipp Mayr ; Leif Azzopardi ; Norbert Fuhr ; Claudia Hauff ; Djoerd Hiemstra. Springer Verlag, 2019. pp. 309-315 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{a5a3250d16894dfab62f1851779d4889,
title = "CheckThat! at CLEF 2019: Automatic identification and verification of claims",
abstract = "We introduce the second edition of the CheckThat! Lab, part of the 2019 Cross-Language Evaluation Forum (CLEF). CheckThat! proposes two complementary tasks. Task 1: predict which claims in a political debate should be prioritized for fact-checking. Task 2: rank Web-retrieved pages against a check-worthy claim based on their usefulness for fact-checking, extract useful passages from those pages, and then use them all to decide whether the claim is factually true or false. Checkthat! provides a full evaluation framework, consisting of data in English (derived from fact-checking sources) and Arabic (gathered and annotated from scratch) and evaluation based on mean average precision (MAP) for ranking and F 1 for classification tasks.",
author = "Tamer Elsayed and Preslav Nakov and Alberto Barron and Maram Hasanain and Reem Suwaileh and Giovanni Martino and Pepa Atanasova",
year = "2019",
month = "1",
day = "1",
doi = "10.1007/978-3-030-15719-7_41",
language = "English",
isbn = "9783030157180",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "309--315",
editor = "Benno Stein and Philipp Mayr and Leif Azzopardi and Norbert Fuhr and Claudia Hauff and Djoerd Hiemstra",
booktitle = "Advances in Information Retrieval - 41st European Conference on IR Research, ECIR 2019, Proceedings",

}

TY - GEN

T1 - CheckThat! at CLEF 2019

T2 - Automatic identification and verification of claims

AU - Elsayed, Tamer

AU - Nakov, Preslav

AU - Barron, Alberto

AU - Hasanain, Maram

AU - Suwaileh, Reem

AU - Martino, Giovanni

AU - Atanasova, Pepa

PY - 2019/1/1

Y1 - 2019/1/1

N2 - We introduce the second edition of the CheckThat! Lab, part of the 2019 Cross-Language Evaluation Forum (CLEF). CheckThat! proposes two complementary tasks. Task 1: predict which claims in a political debate should be prioritized for fact-checking. Task 2: rank Web-retrieved pages against a check-worthy claim based on their usefulness for fact-checking, extract useful passages from those pages, and then use them all to decide whether the claim is factually true or false. Checkthat! provides a full evaluation framework, consisting of data in English (derived from fact-checking sources) and Arabic (gathered and annotated from scratch) and evaluation based on mean average precision (MAP) for ranking and F 1 for classification tasks.

AB - We introduce the second edition of the CheckThat! Lab, part of the 2019 Cross-Language Evaluation Forum (CLEF). CheckThat! proposes two complementary tasks. Task 1: predict which claims in a political debate should be prioritized for fact-checking. Task 2: rank Web-retrieved pages against a check-worthy claim based on their usefulness for fact-checking, extract useful passages from those pages, and then use them all to decide whether the claim is factually true or false. Checkthat! provides a full evaluation framework, consisting of data in English (derived from fact-checking sources) and Arabic (gathered and annotated from scratch) and evaluation based on mean average precision (MAP) for ranking and F 1 for classification tasks.

UR - http://www.scopus.com/inward/record.url?scp=85064878114&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85064878114&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-15719-7_41

DO - 10.1007/978-3-030-15719-7_41

M3 - Conference contribution

SN - 9783030157180

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 309

EP - 315

BT - Advances in Information Retrieval - 41st European Conference on IR Research, ECIR 2019, Proceedings

A2 - Stein, Benno

A2 - Mayr, Philipp

A2 - Azzopardi, Leif

A2 - Fuhr, Norbert

A2 - Hauff, Claudia

A2 - Hiemstra, Djoerd

PB - Springer Verlag

ER -