Explainability of the Implications of Supervised and Unsupervised Face Image Quality Estimations Through Activation Map Variation Analyses in Face Recognition Models

It is challenging to derive explainability for unsupervised or\nstatistical-based face image quality assessment (FIQA) methods. In this work,\nwe propose a novel set of explainability tools to derive reasoning for\ndifferent FIQA decisions and their face recognition (FR) performance\nimplications. We avoid limiting the deployment of our tools to certain FIQA\nmethods by basing our analyses on the behavior of FR models when processing\nsamples with different FIQA decisions. This leads to explainability tools that\ncan be applied for any FIQA method with any CNN-based FR solution using\nactivation mapping to exhibit the network's activation derived from the face\nembedding. To avoid the low discrimination between the general spatial\nactivation mapping of low and high-quality images in FR models, we build our\nexplainability tools in a higher derivative space by analyzing the variation of\nthe FR activation maps of image sets with different quality decisions. We\ndemonstrate our tools and analyze the findings on four FIQA methods, by\npresenting inter and intra-FIQA method analyses. Our proposed tools and the\nanalyses based on them point out, among other conclusions, that high-quality\nimages typically cause consistent low activation on the areas outside of the\ncentral face region, while low-quality images, despite general low activation,\nhave high variations of activation in such areas. Our explainability tools also\nextend to analyzing single images where we show that low-quality images tend to\nhave an FR model spatial activation that strongly differs from what is expected\nfrom a high-quality image where this difference also tends to appear more in\nareas outside of the central face region and does correspond to issues like\nextreme poses and facial occlusions. The implementation of the proposed tools\nis accessible here [link].\n