Separating directional lighting variability in statistical face modelling based on texture space decomposition
View Patent ↗A technique for determining a characteristic of a face or certain other object within a scene captured in a digital image including acquiring an image and applying a linear texture model that is constructed based on a training data set and that includes a class of objects including a first subset of model components that exhibit a dependency on directional lighting variations and a second subset of model components which are independent of directional lighting variations. A fit of the model to the face or certain other object is obtained including adjusting one or more individual values of one or more of the model components of the linear texture model. Based on the obtained fit of the model to the face or certain other object in the scene, a characteristic of the face or certain other object is determined.
1. A method for constructing a linear texture model of a class of objects, comprising a first subset of model components that exhibit a dependency on directional lighting variations and a second subset of model components that are independent of directional lighting variations, comprising:
providing a training set including a plurality of object images wherein various instances of each object cover a range of directional lighting conditions;
applying to the images a linear texture model constructed from object images each captured under uniform lighting conditions and forming a uniform lighting subspace (ULS);
determining a set of residual texture components between object images captured under directional lighting conditions and said linear texture model constructed from object images each captured under uniform lighting conditions;
constructing an orthogonal texture subspace from said residual texture components to form a directional lighting subspace (DLS); and
combining said uniform lighting subspace (ULS) with said directional lighting subspace (DLS) to form a new linear texture model.
2. The method of claim 1 , further comprising:
(i) applying the new linear texture model that is constructed based on a training data set and comprises a class of objects including a subset of model components that has uniform lighting subspace (ULS) and directional lighting subspace (DLS) components,
(ii) obtaining a fit of said new model to said face or certain other object in the scene including adjusting one or more individual values of one or more model components of said new linear texture model;
(iii) based on the obtained fit of the new model to said face or certain other object in the scene, determining a characteristic of the face or certain other object, and
(iv) electronically storing, transmitting, applying a face or other object recognition program to, editing, or displaying the corrected face image or certain other object including the determined characteristic, or combinations thereof.
3. The method of claim 2 , further comprising:
(v) further comprising changing one or more values of one or more model components of the new linear texture model to generate a further adjusted object model;
(vi) obtaining a fit of said new model to said face or certain other object in the scene including adjusting one or more individual values of one or more model components of said new linear texture model; and
(vii) based on the obtained fit of the new model to said face or certain other object in the scene, determining a characteristic of the face or certain other object.
4. The method of claim 1 , wherein the model components comprise eigenvectors, and the individual values comprises eigenvalues of the eigenvectors.
5. One or more non-transitory processor-readable media having code embodied therein for programming a processor to perform a method for constructing a linear texture model of a class of objects, comprising a first subset of model components that exhibit a dependency on directional lighting variations and a second subset of model components that are independent of directional lighting variations, the method comprising:
(a) providing a training set including a plurality of object images wherein various instances of each object cover a range of directional lighting conditions;
(b) applying to the images a linear texture model constructed from object images each captured under uniform lighting conditions and forming a uniform lighting subspace (ULS);
(c) determining a set of residual texture components between object images captured under directional lighting conditions and said linear texture model constructed from object images each captured under uniform lighting conditions;
(d) constructing an orthogonal texture subspace from said residual texture components to form a directional lighting subspace (DLS); and
(e) combining said uniform lighting subspace (ULS) with said directional lighting subspace (DLS) to form a new linear texture model.
6. The one or more non-transitory processor-readable media of claim 5 , the method further comprising:
(i) applying the new linear texture model that is constructed based on a training data set and comprises a class of objects including a subset of model components that has uniform lighting subspace (ULS) and directional lighting subspace (DLS) components,
(ii) obtaining a fit of said new model to said face or certain other object in the scene including adjusting one or more individual values of one or more model components of said new linear texture model;
(iii) based on the obtained fit of the new model to said face or certain other object in the scene, determining a characteristic of the face or certain other object, and
(iv) electronically storing, transmitting, applying a face or other object recognition program to, editing, or displaying the corrected face image or certain other object including the determined characteristic, or combinations thereof.
7. The one or more non-transitory processor-readable media of claim 6 , the method further comprising:
(v) further comprising changing one or more values of one or more model components of the new linear texture model to generate a further adjusted object model;
(vi) obtaining a fit of said new model to said face or certain other object in the scene including adjusting one or more individual values of one or more model components of said new linear texture model; and
(vii) based on the obtained fit of the new model to said face or certain other object in the scene, determining a characteristic of the face or certain other object.
8. The one or more non-transitory processor-readable media of claim 5 , wherein the model components comprise eigenvectors, and the individual values comprises eigenvalues of the eigenvectors.
9. A digital image acquisition device including an optoelectonic system for acquiring a digital image, and a digital memory having stored therein processor-readable code for programming the processor to perform a method for constructing a linear texture model of a class of objects, comprising a first subset of model components that exhibit a dependency on directional lighting variations and a second subset of model components that are independent of directional lighting variations, wherein the method comprises:
(a) providing a training set including a plurality of object images wherein various instances of each object cover a range of directional lighting conditions;
(b) applying to the images a linear texture model constructed from object images each captured under uniform lighting conditions and forming a uniform lighting subspace (ULS);
(c) determining a set of residual texture components between object images captured under directional lighting conditions and said linear texture model constructed from object images each captured under uniform lighting conditions;
(d) constructing an orthogonal texture subspace from said residual texture components to form a directional lighting subspace (DLS); and
(e) combining said uniform lighting subspace (ULS) with said directional lighting subspace (DLS) to form a new linear texture model.
10. The device of claim 9 , wherein the method further comprises:
(i) applying the new linear texture model that is constructed based on a training data set and comprises a class of objects including a subset of model components that has uniform lighting subspace (ULS) and directional lighting subspace (DLS) components,
(ii) obtaining a fit of said new model to said face or certain other object in the scene including adjusting one or more individual values of one or more model components of said new linear texture model;
(iii) based on the obtained fit of the new model to said face or certain other object in the scene, determining a characteristic of the face or certain other object, and
(iv) electronically storing, transmitting, applying a face or other object recognition program to, editing, or displaying the corrected face image or certain other object including the determined characteristic, or combinations thereof.
11. The device of claim 10 , wherein the method further comprises:
(v) further comprising changing one or more values of one or more model components of the new linear texture model to generate a further adjusted object model;
(vi) obtaining a fit of said new model to said face or certain other object in the scene including adjusting one or more individual values of one or more model components of said new linear texture model; and
(vii) based on the obtained fit of the new model to said face or certain other object in the scene, determining a characteristic of the face or certain other object.
12. The device of claim 9 , wherein the model components comprise eigenvectors, and the individual values comprises eigenvalues of the eigenvectors.