Representing faces: local and holistic approaches with application to recognition

Face analysis from 2D images and videos is a central task in many computer vision applications. Methods developed to this end perform either face recognition or facial expression recognition, and in both cases, results are negatively influenced by variations in pose, illumination, and resolution of the face. Such variations have a lower impact on 3D face data, which has given the way to the idea of using a 3D Morphable Model as an intermediate tool to enhance face analysis on 2D data. In the first part of this thesis, a new approach for constructing a 3D Morphable Shape Model (called DL-3DMM) is proposed. It is shown that this solution can reach the accuracy of deformation required in applications where fine details of the face are concerned. The DL-3DMM is then exploited to develop a new and effective frontalization algorithm, which can produce a frontal facing view of unconstrained face images. The rendered frontal views result artifact-free and pixelwise aligned so that matching consistency between local descriptors is enhanced. Results obtained with this approach are comparable with the state-of-the-art. Lately, in contrast to local descriptors based approaches, methods grounded on deep learning algorithms proved to be dramatically effective for face recognition in the wild. It has been extensively demonstrated that methods exploiting Deep Convolutional Neural Networks (DCNN) are powerful enough to overcome to a great extent many problems that negatively affected computer vision algorithms based on hand-crafted features. The DCNNs excellent discriminative power comes from the fact that they learn low- and high-level representations directly from the raw image data. Considering this, it can be assumed that the performance of a DCNN is influenced by the characteristics of the raw image data that are fed to the network. In the final part of this thesis, the effects of different raw data characteristics on face recognition using well known DCNN architectures are presented.