3D Facial Matching by Spiral Convolutional Metric Learning and a Biometric Fusion-Net of Demographic Properties

Face recognition is a widely accepted biometric verification tool, as the\nface contains a lot of information about the identity of a person. In this\nstudy, a 2-step neural-based pipeline is presented for matching 3D facial shape\nto multiple DNA-related properties (sex, age, BMI and genomic background). The\nfirst step consists of a triplet loss-based metric learner that compresses\nfacial shape into a lower dimensional embedding while preserving information\nabout the property of interest. Most studies in the field of metric learning\nhave only focused on 2D Euclidean data. In this work, geometric deep learning\nis employed to learn directly from 3D facial meshes. To this end, spiral\nconvolutions are used along with a novel mesh-sampling scheme that retains\nuniformly sampled 3D points at different levels of resolution. The second step\nis a multi-biometric fusion by a fully connected neural network. The network\ntakes an ensemble of embeddings and property labels as input and returns\ngenuine and imposter scores. Since embeddings are accepted as an input, there\nis no need to train classifiers for the different properties and available data\ncan be used more efficiently. Results obtained by a 10-fold cross-validation\nfor biometric verification show that combining multiple properties leads to\nstronger biometric systems. Furthermore, the proposed neural-based pipeline\noutperforms a linear baseline, which consists of principal component analysis,\nfollowed by classification with linear support vector machines and a Naive\nBayes-based score-fuser.\n