Multilinear analysis of face image ensembles

Abstract

Machine based face recognition is an important area of research that has attracted significant attention over the past few decades. Recently, multilinear models of face images have gained prominence as an alternative method for face recognition. Against linear techniques, multilinear models offer the advantage of having more complex models. Against kernel and manifold based non-linear techniques, the advantage lies in having more intuitive and computationally frugal modelling. In this thesis, we present an in-depth analysis and understanding of different properties associated with multilinear analysis and propose three different face recognition algorithms and a unified framework addressing open issues in face recognition.We first propose a face recognition algorithm primarily to address the limitations of the existing multilinear based algorithms in the form of their inability to handle test images that are in unseen conditions. The algorithm is based on the construction of a new representational basis multilinear eigenmodes, enabling representation and classification of faces at unseen conditions. Subsequently, we propose a second algorithm to address the high computational complexity of the first algorithm. We define a set of person-specific bases to represent person-specific images under all variations, and based on this propose an efficient recognition algorithm. Next, we propose a framework of face recognition based on an interpretation of the multilinear analysis as a factor analysis paradigm. We the reformulate all the multilinear based algorithms to link them to a single optimization framework. A theoretical comparison of these algorithms is performed revealing the fundamental differences between them and their applicability in different face recognition scenarios. Experiments performed to compare multilinear analysis based methods to the leading linear and non-linear techniques reveal the superiority of our second algorithm on both measures of recognition accuracy and test speed.Next, we address the issue of inadequate training samples that arise in many application scenarios. We introduce a novel “friendly-hostile” paradigm, in which we propose a mechanism to compensate for the low number of training samples of hostile people by learning the structure of face images from a large training set of the friendly people. The formulation is built on a novel synthesis paradigm that is based on the unique factorization properties of the multilinear analysis. Experimental results show significant performance gain in comparison to the conventional methods. We also discuss issues concerning unbalanced datasets, wherein some people may be under-represented than others in the training set. This results in different apriori bias per class, affecting conventional recognition algorithms. Based on theory and experiments we demonstrate that our algorithm does not get affected by any such imbalance in bias and produces consistent performance in all situations.