Your fairness may vary: Pretrained language model fairness in toxic text classification

We study the performance-fairness trade-off in more than a dozen fine-tuned LMs for toxic text classification. We empirically show that no blanket statement can be made with respect to the bias of large versus regular versus compressed models. Moreover, we find that focusing on fairness-agnostic performance metrics can lead to models with varied fairness characteristics.

cs.CL cs.LG Adversarial Robustness in Machine Learning Hate Speech and Cyberbullying Detection Topic Modeling Computer Science Linguistics

Paper

Similar papers

Your fairness may vary: Pretrained language model fairness in toxic text classificationIoana Baldini, Dennis Wei, Karthikeyan Natesan Ramamurthy et al.2022 · Findings of the Association for Computational Linguistics: ACL 2022 In Library

Finding Pareto Trade-offs in Fair and Accurate Detection of Toxic SpeechSoumyajit Gupta, Venelin Kovatchev, Das, Anubrata et al.2022 · arXiv (Cornell University) In Library

On Bias and Fairness in NLP: Investigating the Impact of Bias and Debiasing in Language Models on the Fairness of Toxicity DetectionFatma Elsafoury, Stamos Katsigiannis2023 · arXiv (Cornell University) · 7 citations In Library

Counterfactual Fairness in Text Classification through RobustnessSahaj Garg, Vincent Perot, Nicole Limtiaco et al.2018 · AAAI/ACM Conference on AI, Ethics, and Society · 294 citations In Library

When to Invoke: Refining LLM Fairness with Toxicity AssessmentJing Ren, Bowen Li, Ziqi Xu et al.2026 · The Web Conference · 3 citations In Library

All Should Be Equal in the Eyes of Language Models: Counterfactually Aware Fair Text GenerationPragyan Banerjee, Abhinav Java, Surgan Jandial et al.2023 · arXiv (Cornell University) · 5 citations In Library

Towards Procedural Fairness: Uncovering Biases in How a Toxic Language Classifier Uses Sentiment InformationIsar Nejadgholi, Esma Balkır, Kathleen Fraser et al.2022 In Library

Towards Procedural Fairness: Uncovering Biases in How a Toxic Language Classifier Uses Sentiment InformationIsar Nejadgholi, Esma Balkır, Kathleen Fraser et al.2022 · arXiv (Cornell University) In Library

Large Language Models Still Exhibit Bias in Long TextWonje Jeung, Dongjae Jeon, Ashkan Yousefpour et al.2024 · arXiv (Cornell University) · 16 citations In Library

Large Language Models Still Exhibit Bias in Long TextWonje Jeung, Dongjae Jeon, Ashkan Yousefpour et al.2025 In Library

Source of record

arXiv

44B

Welcome.

This panel sticks with you. Pick anything below and it opens right beside it, so you can dig through 60,000-plus records without ever losing your spot here.

Travel 44B

LibraryPapers, policy, standards, statute LabsEvery organization building AI ModelsIntelligence, price, and speed BenchmarksThe evaluation catalog IncidentsAI failures and harms SearchOne field across everything DashboardThe state of AI accountability in NY 44B RegistryThe Art. 44-B compliance portal