Quantifying the Extent to Which Race and Gender Features Determine Identity in Commercial Face Recognition Algorithms

Human face features can be used to determine individual identity as well as\ndemographic information like gender and race. However, the extent to which\nblack-box commercial face recognition algorithms (CFRAs) use gender and race\nfeatures to determine identity is poorly understood despite increasing\ndeployments by government and industry. In this study, we quantified the degree\nto which gender and race features influenced face recognition similarity scores\nbetween different people, i.e. non-mated scores. We ran this study using five\ndifferent CFRAs and a sample of 333 diverse test subjects. As a control, we\ncompared the behavior of these non-mated distributions to a commercial iris\nrecognition algorithm (CIRA). Confirming prior work, all CFRAs produced higher\nsimilarity scores for people of the same gender and race, an effect known as\n"broad homogeneity". No such effect was observed for the CIRA. Next, we applied\nprincipal components analysis (PCA) to similarity score matrices. We show that\nsome principal components (PCs) of CFRAs cluster people by gender and race, but\nthe majority do not. Demographic clustering in the PCs accounted for only 10 %\nof the total CFRA score variance. No clustering was observed for the CIRA. This\ndemonstrates that, although CFRAs use some gender and race features to\nestablish identity, most features utilized by current CFRAs are unrelated to\ngender and race, similar to the iris texture patterns utilized by the CIRA.\nFinally, reconstruction of similarity score matrices using only PCs that showed\nno demographic clustering reduced broad homogeneity effects, but also decreased\nthe separation between mated and non-mated scores. This suggests it's possible\nfor CFRAs to operate on features unrelated to gender and race, albeit with\nsomewhat lower recognition accuracy, but that this is not the current\ncommercial practice.\n