Method and apparatus for local binary pattern based facial feature localization
Various methods for local binary pattern based facial feature localization. One example method includes determining an eye state classification of an input image. The example method may also include selecting a texture model for a global shape and an associated mean shape based on eye center positions and the eye state classification, and adjusting locations of feature points defined by the mean shape based on the texture model for the global shape and an associated global shape model. Similar and related example methods and example apparatuses are also provided.
1. A method comprising:
determining an eye state classification of an input image;
selecting, using a processor, a texture model for a global shape and an associated mean shape based on eye center positions and the eye state classification, wherein the mean shape comprises all feature points and the global shape model comprises a face frame based on a plurality of fixed points in the input image, wherein the texture model for the global shape is a local binary pattern model determined based on a probability density; and
adjusting locations of feature points defined by the mean shape based on the texture model for the global shape and an associated global shape model.
2. The method of claim 1 further comprising:
obtaining the input image;
determining initial eye center positions via eye detection; and
normalizing and scaling the input image based on the initial eye center positions.
3. The method of claim 1 , wherein adjusting the locations of the feature points includes adjusting the locations of the feature points based on the associated global shape model, wherein the global shape model is included within a hierarchical shape model.
4. The method of claim 3 , wherein the hierarchical shape model comprises a global shape model, a plurality of local shape models, and a mean shape model.
5. The method of claim 4 , wherein the plurality of local shape models are based on the global shape model and the mean shape model is based on the plurality of local shape models, wherein the local shape models comprises feature points for a respective facial component.
6. The method of claim 1 , wherein selecting the texture model for the global shape includes selecting the texture model from a collection of models for an open eye state and closed eye state.
7. The method of claim 1 further comprising performing a mouth corner detection and a mouth state classification.
8. The method of claim 1 further comprising:
performing a mouth corner detection and a mouth state classification;
selecting a local shape model and an associated local texture model for a given facial component based on a current state of the given component, the associated local texture model being a local binary pattern model determined on a probability density; and
further adjusting the feature points associated with a given facial component based on the local shape model and the associated texture model for the given facial component.
9. An apparatus comprising at least one processor and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to perform:
determine an eye state classification of an input image;
select a texture model for a global shape and an associated mean shape based on eye center positions and the eye state classification, wherein the mean shape comprises all feature points and the global shape model comprises a face frame based on a plurality of fixed points in the input image, wherein the texture model for the global state is a local binary pattern model determined based on a probability density; and
adjust locations of feature points defined by the mean shape based on the texture model for the global shape model and an associated global shape model.
10. The apparatus of claim 9 , wherein the apparatus is further caused to:
obtain the input image;
determine initial eye center positions via eye detection; and
normalize and scale the input image based on the initial eye center positions.
11. The apparatus of claim 9 , wherein the apparatus caused to adjust the locations of the feature points includes adjusting the locations of the feature points based on the associated global shape model, wherein the global shape model is included within a hierarchical shape model.
12. The apparatus of claim 11 , wherein the hierarchical shape model comprises a mean shape model, a global shape model, a local shape model and a mean shape model.
13. The apparatus of claim 12 , wherein the plurality of local shape models are based on the global shape model and the mean shape model is based on the plurality of local shape models, wherein the local shape models comprises feature points for a respective facial component.
14. The apparatus of claim 9 , wherein the apparatus caused to select the texture model for the global shape includes being caused to select the texture model from a collection of models for an open eye state and closed eye state.
15. The apparatus of claim 9 , wherein the apparatus is further caused to perform mouth corner detection and mouth state classification.
16. The apparatus of claim 9 , wherein the apparatus is further caused to:
detect mouth corners and classify a mouth state;
select a local shape model and an associated local texture model for a given facial component based on a current state of the given component, the associated local texture model being a local binary pattern model determined on a probability density; and
further adjust the feature points associated with a given facial component based on the local shape model and the associated texture model for the given facial component.
17. The apparatus of claim 9 , wherein the apparatus comprises a mobile terminal; and wherein the apparatus further comprises user interface circuitry and user interface software configured to facilitate user control of at least some functions of the mobile terminal through the use of a display and configured to respond to user inputs, and a display and display circuitry configured to display at least a portion of a user interface of the mobile terminal, the display and the display circuitry configured to facilitate user control of at least some functions of mobile terminal.
18. The apparatus of claim 17 , further comprising image capturing circuitry configured to capture an image including a face.
19. A computer program product comprising a non-transitory computer readable storage medium having computer program code stored thereon, the computer program code being configured to, when executed, cause an apparatus to perform:
determining an eye state classification of an input image;
selecting a texture model for a global shape and an associated mean shape based on eye center positions and the eye state classification, wherein the mean shape comprises all feature points and the global shape model comprises a face frame based on a plurality of fixed points in the input image, wherein the texture model for the global shape is a local binary pattern model determined based on a probability density; and
adjusting locations of feature points defined by the mean shape based on the texture model for the global shape and an associated shape model.
20. The computer program product of claim 19 , wherein the computer program code is further configured to cause the apparatus to perform:
obtaining the input image;
determining initial eye center positions via eye detection; and
normalizing and scaling the input image based on the initial eye center positions.
21. The computer program product of claim 19 , wherein the computer program code configured to cause the apparatus to perform adjusting the locations of the feature points includes being configured to cause the apparatus to perform adjusting the locations of the feature points based on the associated global shape model, wherein the global shape model is included within a hierarchical shape model.
22. The computer program product of claim 21 , wherein the hierarchical shape model comprises a global shape model, a plurality of local shape models, and a mean shape model.
23. The method of claim 22 , wherein the plurality of local shape models are based on the global shape model and the mean shape model is based on the plurality of local shape models, wherein the local shape models comprises feature points for a respective facial component.
24. The computer program product of claim 19 , wherein the computer program code configured to cause the apparatus to perform selecting the texture model for global shape includes being configured to cause the apparatus to perform selecting the texture model from a collection of models for an open eye state and closed eye state.
25. The computer program product of claim 19 , wherein the computer program code is further configured to cause the apparatus to perform mouth corner detection and mouth state classification.
26. The computer program product of claim 19 , wherein the computer program code is further configured to cause the apparatus to perform:
performing a mouth corner detection and a mouth state classification;
selecting a local shape model and an associated local texture model for a given facial component based on a current state of the given component, the associated local texture model being a local binary pattern model determined on a probability density; and
further adjusting the feature points associated with a given facial component based on the local shape model and the associated texture model for the given facial component.