Face Recognition against Mouth Shape Variations

Abstract

In this paper, face recognition against mouth shape variations is investigated. In order to detect possible mouth variations, the inner mouth landmarks are first detected by a landmark detector and then used to estimate the connectivity between the upper lip and the lower lip of a face image. The vertical distance between the middle inner points of upper lip and the lower lip is calculated, and then used with appropriate threshold to decide whether the two lips are connected or separated. If the two lips are not connected, we further detect the teeth positions based on the colour pixel values, and then a face can be classified into four classes: closed mouth (C), closed mouth with teeth (Ct), open mouth (O), and open mouth with teeth (Ot). Next we attempt to transform face images in classes Ct and Ot into classes C and O respectively, and this is done by shrinking the areas with the upper and lower parts of the mouth by a proposed alignment approach. In this mouth closing process, both face areas located above and below the mouth are changed dramatically and the whole face image is vertically stretched to the original size in order to change the face image into neutral appearance. Extensive experiments on AR database and BU database show that the proposed shape correction approach to closing an opened mouth can significantly increase the recognition rate up to 21.5% by using PCA and 17.5% by using LDA.