We Can Detect Your Bias: Predicting the Political Ideology of News Articles

We explore the task of predicting the leading political ideology or bias of\nnews articles. First, we collect and release a large dataset of 34,737 articles\nthat were manually annotated for political ideology -left, center, or right-,\nwhich is well-balanced across both topics and media. We further use a\nchallenging experimental setup where the test examples come from media that\nwere not seen during training, which prevents the model from learning to detect\nthe source of the target news article instead of predicting its political\nideology. From a modeling perspective, we propose an adversarial media\nadaptation, as well as a specially adapted triplet loss. We further add\nbackground information about the source, and we show that it is quite helpful\nfor improving article-level prediction. Our experimental results show very\nsizable improvements over using state-of-the-art pre-trained Transformers in\nthis challenging setup.\n