Distilling Generative-Discriminative Representations for Very Low-Resolution Face Recognition

Very low-resolution face recognition is challenging due to the serious loss of informative facial details in resolution degradation. Recent approaches based on knowledge distillation provide an effective solution by distilling knowledge from a well-trained teacher for high-resolution face recognition and transferring it to a student for low-resolution face recognition. In general, the existing approaches usually take a discriminative model as teacher, where the teacher knowledge is trained in an abstract manner and provides poor transfer efficiency to compensate for the missing knowledge in low-resolution faces. To make more complete knowledge transfer, we propose a generative-discriminative representation distillation approach that combines generative representation with cross-resolution aligned knowledge distillation. This approach facilitates very low-resolution face recognition by jointly distilling generative and discriminative models via two distillation modules. Firstly, the generative representation distillation takes the encoder of a diffusion model pretrained for face super-resolution as the generative teacher to supervise the learning of the student backbone via feature regression, and then freezes the student backbone. After that, the discriminative representation distillation further considers a pretrained face recognizer as the discriminative teacher to supervise the learning of the student head via cross-resolution relational contrastive distillation. In this way, the general backbone representation can be transformed into discriminative head representation, leading to a robust and discriminative student model for very low-resolution face recognition. Our approach improves the recovery of the missing details in very low-resolution faces and achieves better knowledge transfer. Extensive experiments on face datasets demonstrate that our approach enhances the recognition accuracy of very low-resolution faces, showcasing its effectiveness and adaptability.