Joint boosting feature selection for robust face recognition
View Patent ↗Methods and systems are provided for selecting features that will be used to recognize faces. Three-dimensional models are used to synthesize a database of virtual face images. The virtual face images cover wide appearance variations, different poses, different lighting conditions and expression changes. A joint boosting algorithm is used to identify discriminative features by selecting features from the plurality of virtual images such that the identified discriminative features are independent of the other images included in the database.
1. A computer-implemented method of verifying the identity of individuals including selecting facial features used to recognize faces with a face recognition module operating on a processor, the method comprising:
collecting three-dimensional face models using a computer input scanning device and storing the three-dimensional face models in a hardware memory storage device;
using the face recognition module to perform the following:
create a plurality of electronic virtual images from the three-dimensional face models for each of the faces to create a dataset, the dataset comprising the electronic virtual images being stored in a hardware memory storage device, wherein the processor is operative on the hardware memory storage device;
identify discriminative features by selecting features from the plurality of electronic virtual images such that the identified discriminative features are independent of the dataset, wherein identifying comprises evaluating a performance of features by evaluating:
G
t
c
(
j
)
=
∫
x
∈
X
g
(
h
j
c
,
+
(
x
w
t
c
)
,
h
j
c
,
-
(
x
w
t
c
)
)
ⅆ
x
.
wherein h c j ,+ (x|w) and h c j ,−(x|w) are weighted distributions,
c represents a model class,
t represents a training sample number,
j represents a feature being evaluated,
w t c represents a weight of a c th boosting model, at a t th step,
and function g(r(x),s(x)) is a measure of classification error of a logistic classifier defined on weighted distributions r(x) and s(x),
assign weights to the identified discriminative features based on classification strengths of the features, wherein the discriminative features that have strong classification strengths are assigned large weights,
boost discriminative features having weak classification strengths by assigning a large weight to a combination of discriminative features having weak classification strengths,
receive an unidentified facial image from an input device,
analyze the discriminative features of the unidentified facial image, and
verify the identity of an individual associated with the unidentified facial image using the discriminative features identified; and
providing an indication to the user of the verification, wherein the indication includes a candidate list of matches.
2. The method of claim 1 , wherein creating comprises creating at least 500 virtual images for each of the faces.
3. The method of claim 1 , wherein creating comprises creating at least 600 virtual images for each of the faces.
4. The method of claim 1 , wherein the three-dimensional face models are collected from at least 100 individuals.
5. The method of claim 1 , wherein the three-dimensional face models are collected from a three-dimensional laser-scanner.
6. The method of claim 1 , wherein identifying further comprises determining Bayesian error by evaluating:
R
(
f
t
,
j
c
(
x
)
)
=
∫
x
∈
X
min
[
h
j
c
,
+
(
x
w
t
c
)
,
h
j
c
,
-
(
x
w
t
c
)
]
ⅆ
x
.
7. The method of claim 1 , wherein identifying comprises evaluating a greedy feature selection process.
8. The method of claim 1 , wherein identifying comprises evaluating a joint boosting algorithm.
9. The method of claim 1 , wherein at least some of the virtual images comprises images showing different poses.
10. The method of claim 1 , wherein at least some of the virtual images comprises images having different illumination.
11. The method of claim 1 , wherein at least some of the virtual images comprises images having different facial expressions.
12. The method of claim 1 , wherein identifying comprises identifying weak classifiers by analyzing a multiple bin histogram showing numbers of virtual images that have values for a set of features.
13. The method of claim 12 , wherein identifying further includes selecting several weak classifiers.
14. A computer-implemented method of recognizing facial images with a face recognition module operating on a processor, the method comprising:
creating three-dimensional face models for a group of faces using a computer input scanning device;
creating a plurality of virtual images from the three-dimensional face models for each of the faces to create a dataset, the dataset comprising the virtual images being stored in a hardware memory storage device, wherein the processor is operative on the hardware memory storage device;
identifying discriminative features by selecting features from the plurality of virtual images such that the identified discriminative features are independent of the dataset, wherein identifying comprises evaluating a performance of features by evaluating:
G t c ( j )= ∫ x ∈X g ( h c j ,+ ( x|w t c ), h c j ,− ( x|w t c )) dx
wherein h c j ,+ (x|w) and h c j ,−(x|w) are weighted distributions,
c represents a model class,
t represents a training sample number,
j represents a feature being evaluated,
w t c represents a weight of a c th boosting model, at a t th step, and function g(r(x),s(x)) is a measure of classification error of a logistic classifier defined on weighted distributions r(x) and s(x);
generalizing the selected features to recognize faces from other databases;
analyzing the identified discriminative features of an unknown facial image, wherein the unknown facial image is not part of the dataset;
determining the identity of an individual associated with the unknown facial image using the discriminative features identified; and
providing an indication to a user of the determination, wherein the indication includes a candidate list of matches.
15. The method of claim 14 , wherein analyzing comprises comparing features of the unknown facial image to features of images in the dataset to create a candidate list of matches.
16. The method of claim 14 , wherein at least some of the virtual images comprises images showing different poses.
17. At least one computer-readable storage medium containing computer-executable instructions for causing a computer device including a processor to select facial features used to recognize faces by performing the steps comprising:
(a) creating three-dimensional face models for a group of faces using a computer input scanning device;
(b) creating a plurality of virtual images from the three-dimensional face models for each of the faces to create a dataset, the dataset comprising the virtual images being stored in a computer-readable storage device, wherein the processor is operative on the computer-readable storage device;
(c) identifying discriminative features by selecting features from the plurality of virtual images such that the identified discriminative features are independent of the dataset, wherein identifying comprises evaluating a performance of features by evaluating:
G t c ( j )= ∫ x ∈X g ( h c j ,+ ( x|w t c ), h c j ,− ( x|w t c )) dx
wherein h c j ,+ (x|w) and h c j ,−(x|w) are weighted distributions,
c represents a model class,
t represents a training sample number,
j represents a feature being evaluated,
w t c represents a weight of a c th boosting model, at a t th step, and function g(r(x),s(x)) is a measure of classification error of a logistic classifier defined on weighted distributions r(x) and s(x); and
(d) verifying
verifying the identity of an individual associated with an unknown facial image using the discriminative features identified, wherein the unknown facial image is not part of the dataset.