CheckThat! at CLEF 2020: Enabling the Automatic Identification and Verification of Claims in Social Media

We describe the third edition of the CheckThat! Lab, which is part of the\n2020 Cross-Language Evaluation Forum (CLEF). CheckThat! proposes four\ncomplementary tasks and a related task from previous lab editions, offered in\nEnglish, Arabic, and Spanish. Task 1 asks to predict which tweets in a Twitter\nstream are worth fact-checking. Task 2 asks to determine whether a claim posted\nin a tweet can be verified using a set of previously fact-checked claims. Task\n3 asks to retrieve text snippets from a given set of Web pages that would be\nuseful for verifying a target tweet's claim. Task 4 asks to predict the\nveracity of a target tweet's claim using a set of Web pages and potentially\nuseful snippets in them. Finally, the lab offers a fifth task that asks to\npredict the check-worthiness of the claims made in English political debates\nand speeches. CheckThat! features a full evaluation framework. The evaluation\nis carried out using mean average precision or precision at rank k for ranking\ntasks, and F1 for classification tasks.\n

Incidents & harms Language & NLP Misinformation & media ML methods cs.CL cs.AI cs.IR cs.LG Hate Speech and Cyberbullying Detection Misinformation and Its Impacts Topic Modeling

Paper

Similar papers

Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social MediaAlberto Barrón‐Cedeño, Tamer Elsayed, Preslav Nakov et al.2020 · arXiv (Cornell University) In Library

Overview of the CLEF-2019 CheckThat!: Automatic Identification and Verification of ClaimsTamer Elsayed, Preslav Nakov, Alberto Barrón‐Cedeño et al.2021 · arXiv (Cornell University) In Library

Overview of the CLEF--2021 CheckThat! Lab on Detecting Check-Worthy Claims, Previously Fact-Checked Claims, and Fake NewsShaden Shaar, Fatima Haouari, Watheq Mansour et al.2021 · Qatar University QSpace (Qatar University) In Library

Overview of the CLEF-2021 CheckThat! Lab Task 1 on check-worthiness estimation in tweets and political debatesShaden Shaar, Maram Hasanain, Bayan Hamdan et al.2021 · Qatar University QSpace (Qatar University) In Library

Leveraging Social Discourse to Measure Check-worthiness of Claims for Fact-checkingMegha Sundriyal, Md Shad Akhtar, Tanmoy Chakraborty2023 · arXiv (Cornell University) In Library

Z-Index at CheckThat! Lab 2022: Check-Worthiness Identification on Tweet TextPrerona Tarannum, Firoj Alam, Md. Arid Hasan et al.2022 · arXiv (Cornell University) In Library

CrowdChecked: Detecting Previously Fact-Checked Claims in Social MediaMomchil Hardalov, Anton Chernyavskiy, Ivan Koychev et al.2022 · arXiv (Cornell University) In Library

Team Alex at CLEF CheckThat! 2020: Identifying Check-Worthy Tweets With Transformer ModelsAlex Nikolov, Giovanni Da San Martino, Ivan Koychev et al.2020 · arXiv (Cornell University) · 23 citations In Library

Accenture at CheckThat! 2020: If you say so: Post-hoc fact-checking of claims using transformer-based modelsEvan Williams, Paul Rodrigues, Valerie Novak2020 · arXiv (Cornell University) In Library

Accenture at CheckThat! 2021: Interesting claim identification and ranking with contextually sensitive lexical training data augmentationEvan R. Williams, Paul Rodrigues, Sieu L. Tran2021 · arXiv (Cornell University) · 22 citations In Library

Source of record

arXiv

44B

Welcome.

This panel sticks with you. Pick anything below and it opens right beside it, so you can dig through 60,000-plus records without ever losing your spot here.

Travel 44B

LibraryPapers, policy, standards, statute LabsEvery organization building AI ModelsIntelligence, price, and speed BenchmarksThe evaluation catalog IncidentsAI failures and harms SearchOne field across everything DashboardThe state of AI accountability in NY 44B RegistryThe Art. 44-B compliance portal