Machine Learning Fairness in Justice Systems: Base Rates, False Positives, and False Negatives

Machine learning best practice statements have proliferated, but there is a\nlack of consensus on what the standards should be. For fairness standards in\nparticular, there is little guidance on how fairness might be achieved in\npractice. Specifically, fairness in errors (both false negatives and false\npositives) can pose a problem of how to set weights, how to make unavoidable\ntradeoffs, and how to judge models that present different kinds of errors\nacross racial groups. This paper considers the consequences of having higher\nrates of false positives for one racial group and higher rates of false\nnegatives for another racial group. The paper examines how different errors in\njustice settings can present problems for machine learning applications, the\nlimits of computation for resolving tradeoffs, and how solutions might have to\nbe crafted through courageous conversations with leadership, line workers,\nstakeholders, and impacted communities.\n