On Local Features for Face Verification

{W}e compare four local feature extraction techniques for the task of face verification, namely (ordered in terms of complexity): raw pixels, raw pixels with mean removal, 2D Discrete Cosine Transform (DCT) and local Principal Component Analysis (PCA). The comparison is performed in terms of discrimination ability and robustness to illumination changes. We also evaluate the effectiveness of several approaches to modifying standard feature extraction methods in order to increase performance and robustness to illumination changes. Results on the XM2VTS database suggest that when using a Gaussian Mixture Model (GMM) based classifier, the raw pixel technique provides poor discrimination and is easily affected by illumination changes; the mean removed raw pixel technique provides performance that is fairly close to 2D DCT and local PCA, but is considerably affected by illumination changes. The performance of 2D DCT and local PCA techniques is quite similar, suggesting that the 2D DCT technique is to be preferred over the local PCA technique, due to the lower complexity of the 2D DCT. Both 2D DCT and local PCA techniques are considerably more robust to illumination changes compared to the raw pixel techniques. Modifying the 2D DCT and local PCA techniques by removing the first coefficient, which is deemed to be the most affected by illumination changes, clearly enhances robustness; removing more than the first coefficient causes a noticeable reduction in performance on clean images and provides no further gains in robustness. Compared to just throwing out the first coefficient, the use of deltas can achieve a small increase in performance and robustness. Lastly, we suggest that it is more appropriate to use analysis blocks of size 8x8 (as opposed to 16x16) with 2D DCT decomposition; out of the 64 resulting coefficients, the second through to 21-st (resulting in 20 dimensional feature vectors) are the most robust to illumination changes while providing good discriminatory information.