How Does Gender Balance In Training Data Affect Face Recognition Accuracy?

Deep learning methods have greatly increased the accuracy of face\nrecognition, but an old problem still persists: accuracy is usually higher for\nmen than women. It is often speculated that lower accuracy for women is caused\nby under-representation in the training data. This work investigates female\nunder-representation in the training data is truly the cause of lower accuracy\nfor females on test data. Using a state-of-the-art deep CNN, three different\nloss functions, and two training datasets, we train each on seven subsets with\ndifferent male/female ratios, totaling forty two trainings, that are tested on\nthree different datasets. Results show that (1) gender balance in the training\ndata does not translate into gender balance in the test accuracy, (2) the\n"gender gap" in test accuracy is not minimized by a gender-balanced training\nset, but by a training set with more male images than female images, and (3)\ntraining to minimize the accuracy gap does not result in highest female, male\nor average accuracy\n