Colour invariants under a non-linear photometric camera model and their application to face recognition from video

Illumination invariance remains one of the most researched, yet the most challenging aspect of automatic face recognition. In this paper the discriminative power of colour-based invariants is investigated in the presence of large illumination changes between training and query data, when appearance changes due to cast shadows and non-Lambertian effects are significant. Specifically, there are three main contributions: (i) a general photometric model of the camera is described and it is shown how its parameters can be estimated from realistic video input of pseudo-random head motion, (ii) several novel colour-based face invariants are derived for different special instances of the camera model, and (iii) the performance of the largest number of colour-based representations in the literature is evaluated and analysed on a database of 700 video sequences. The reported results suggest that: (i) colour invariants do have a substantial discriminative power which may increase the robustness and accuracy of recognition from low resolution images in extreme illuminations, and (ii) that the non-linearities of the general photometric camera model have a significant effect on recognition performance. This highlights the limitations of previous work and emphasizes the need to assess face recognition performance using training and query data which had been captured by different acquisition equipment.