Performative Castelessness or Ignorant Castefulness? Influence of Mahalanobis' Caste-Distance in Responsible AI Frameworks
This paper analyzes two critical case studies in the field of AI fairness and responsible AI by examining AI Fairness 360, an open-source AI audit framework, and India's Responsible AI strategy to argue that these Responsible AI frameworks function from an external lens. They evaluate AI algorithms and its impact extrinsically without raising internal questions about the politics hidden within fairness mechanisms and assumptions behind Responsible AI. The paper draws from the literature on AI bias and fairness to point out that sociotechnical AI systems studied in this paper execute castefulness, or showcase caste bias, while performing castelessness. The paper analyzes AI Fairness 360, an open-source toolkit to check and mitigate biases in AI algorithms alongside casteist historical roots of Mahalanobis distance, a fairness metric used in this toolkit, to showcase the performance of castelessness and ignorance of castefulness by common practices in AI fairness strategies. The paper stresses this argument by undertaking a speculative exercise of fairness evaluation and mitigation for linear discriminant algorithm amidst the implicit assumptions on which it was formed and sustained. By exploring historical roots of caste profiling and classification tools, the paper establishes a close relationship between a measure of caste distance/difference and a metric used to measure algorithmic fairness. Using this understanding, the paper analyzes India's responsible AI strategy and its branding of #AIforALL to question whose perspectives and voices are included in this endeavor. Thus, this paper unpacks the assumptions behind algorithmic fairness and responsible AI in two critical case studies to showcase that checking and mitigating caste bias is a complex process. The dynamic nature of caste, as deeply woven into India's social fabric, means algorithmic fairness for caste requires questioning assumptions behind all tools, logics, code, and frameworks currently being used for AI systems. The contribution of this paper is to (i) trace the genealogical origins of a specific fairness metric in casteist and eugenicist anthropometry, (ii) analyze its incorporation into a widely used fairness toolkit without acknowledgment of that history, and (iii) connect this to a critique of India's Responsible AI strategy and a facial recognition system following its principles.
Paper
Full text
Performative Castelessness or Ignorant Castefulness? Influence of Mahalanobis' Caste-Distance in Responsible AI Frameworks
Semantic Scholar · Computer Science · 2026
Abstract
This paper analyzes two critical case studies in the field of AI fairness and responsible AI by examining AI Fairness 360, an open-source AI audit framework, and India's Responsible AI strategy to argue that these Responsible AI frameworks function from an external lens. They evaluate AI algorithms and its impact extrinsically without raising internal questions about the politics hidden within fairness mechanisms and assumptions behind Responsible AI. The paper draws from the literature on AI bias and fairness to point out that sociotechnical AI systems studied in this paper execute castefulness, or showcase caste bias, while performing castelessness. The paper analyzes AI Fairness 360, an open-source toolkit to check and mitigate biases in AI algorithms alongside casteist historical roots of Mahalanobis distance, a fairness metric used in this toolkit, to showcase the performance of castelessness and ignorance of castefulness by common practices in AI fairness strategies. The paper stresses this argument by undertaking a speculative exercise of fairness evaluation and mitigation for linear discriminant algorithm amidst the implicit assumptions on which it was formed and sustained. By exploring historical roots of caste profiling and classification tools, the paper establishes a close relationship between a measure of caste distance/difference and a metric used to measure algorithmic fairness. Using this understanding, the paper analyzes India's responsible AI strategy and its branding of #AIforALL to question whose perspectives and voices are included in this endeavor. Thus, this paper unpacks the assumptions behind algorithmic fairness and responsible AI in two critical case studies to showcase that checking and mitigating caste bias is a complex process. The dynamic nature of caste, as deeply woven into India's social fabric, means algorithmic fairness for caste requires questioning assumptions behind all tools, logics, code, and frameworks currently being used for AI systems. The contribution of this paper is to (i) trace the genealogical origins of a specific fairness metric in casteist and eugenicist anthropometry, (ii) analyze its incorporation into a widely used fairness toolkit without acknowledgment of that history, and (iii) connect this to a critique of India's Responsible AI strategy and a facial recognition system following its principles.