Accurate and robust face recognition from RGB-D images with a deep learning approach

Abstract

Face recognition from RGB-D images utilizes 2 complementary types of image data, i.e. colour and depth images, to achieve more accurate recognition. In this paper, we propose a face recognition system based on deep learning, which can be used to verify and identify a subject from the colour and depth face images captured with a consumer-level RGB-D camera. To recognize faces with colour and depth information, our system contains 3 parts: depth image recovery, deep learning for feature extraction, and joint classification. To alleviate the problem of the limited size of available RGB-D data for deep learning, our deep network is firstly trained with colour face dataset, and later fine-tuned on depth face images for transfer learning. Our experiments on some public and our own RGB-D face datasets show that the proposed face recognition system provides very accurate face recognition results and it is robust against variations in head rotation and environmental illumination.