Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social Media

We present an overview of the third edition of the CheckThat! Lab at CLEF\n2020. The lab featured five tasks in two different languages: English and\nArabic. The first four tasks compose the full pipeline of claim verification in\nsocial media: Task 1 on check-worthiness estimation, Task 2 on retrieving\npreviously fact-checked claims, Task 3 on evidence retrieval, and Task 4 on\nclaim verification. The lab is completed with Task 5 on check-worthiness\nestimation in political debates and speeches. A total of 67 teams registered to\nparticipate in the lab (up from 47 at CLEF 2019), and 23 of them actually\nsubmitted runs (compared to 14 at CLEF 2019). Most teams used deep neural\nnetworks based on BERT, LSTMs, or CNNs, and achieved sizable improvements over\nthe baselines on all tasks. Here we describe the tasks setup, the evaluation\nresults, and a summary of the approaches used by the participants, and we\ndiscuss some lessons learned. Last but not least, we release to the research\ncommunity all datasets from the lab as well as the evaluation scripts, which\nshould enable further research in the important tasks of check-worthiness\nestimation and automatic claim verification.\n