Feature point location determination method and apparatus
A method for determining feature point locations in an image performs a first search in a predetermined first search area to search for locations of plural feature points in the image, corrects the locations of the plural feature points based on a geometric layout relationship among the plural feature points searched for, sets a second search area based on the corrected location of each of the feature points, and performs a second search in the second search area to search for the location of each of the feature points. Then, the method determines reliability of the location of each feature point searched for by the second search and selects one of the corrected location and the location searched for by the second search, as a location of the feature point.
1. A method implemented by a computer for determining locations of predetermined feature points, the method comprising:
a first search step of searching a predetermined first search area in an image for a first location of each of the predetermined feature points;
a correction step of correcting the first location of each of the predetermined feature points searched out of the first search area based on a geometric restriction among the predetermined feature points;
a setting step of setting a second search area for each of the predetermined feature points based on the first location of that feature point corrected in said correction step;
a second search step of searching the second search area for a second location of each of the predetermined feature points;
a determination step of determining reliability of the second location of each of the predetermined feature points searched out of the second search area; and
a selection step of selecting, as a final location for each of the predetermined feature points, either the first location corrected in said correction step or the second location searched out of the second search area, based on the reliability of the second location, respectively.
2. The method according to claim 1 , wherein in said determination step, the reliability of the second location of each of the predetermined feature points searched out of the second search area is determined by a calculation based on reliability distribution in the second search area.
3. The method according to claim 1 , further comprising a distance calculation step of calculating, for each of the predetermined feature points, distance between the first location before correction by said location correction step and the first location after the correction,
wherein in said selection step, either the first location corrected in said location correction step or the second location searched out of the second search area is selected, regarding each of the predetermined feature points, based on the distance calculated in said distance calculation step.
4. The method according to claim 1 , wherein the second search area is smaller than the first search area.
5. The method according to claim 2 , further comprising a distance calculation step of calculating, for each of the predetermined feature points, distance between the first location before correction by said location correction step and the first location after the correction,
wherein in said selection step, either the first location corrected in said location correction step or the second location searched out of the second search area is selected, regarding each of the predetermined feature points, based on the distance calculated in said distance calculation step and the reliability calculated in said reliability calculation step.
6. The method according to claim 1 , further comprising a generation step of generating a reliability map which represents a distribution of reliability of existence of each of the predetermined feature points in the image,
wherein in said first search step, the search for the first location of each of the predetermined feature points is performed based on the reliability map.
7. The method according to claim 6 , wherein in said second search step, the search for the second location of each of the predetermined feature points is performed using the image.
8. The method according to claim 3 , further comprising a change step of changing size of the second search area according to the distance calculated by said distance calculation step.
9. The method according to claim 1 , further comprising a change step of changing the second search area for each type of feature point to be localized.
10. The method according to claim 1 , wherein in said correction step, a projection vector is generated by projecting an input vector whose elements are made up of coordinate values of the predetermined feature points onto an eigenspace using a pre-generated eigenvector, the projection vector is inverse-projected generated using the eigenvector, and thereby each of the first location of the predetermined feature points are corrected.
11. The method according to claim 10 , wherein the eigenvector is generated by principal component analysis.
12. The method according to claim 1 , wherein the feature points to be localized are facial organs of an animal or characteristic parts of an organ.
13. An apparatus for determining locations of predetermined feature points, the apparatus comprising:
a computer adapted to perform the following units:
a first search unit configured to search a predetermined first search area in an image for a first location of each of the predetermined feature points;
a correction unit configured to correct the first location of each of the predetermined feature points searched out of the first search area based on a geometric restriction among the predetermined feature points;
a setting unit configured to set a second search area for each of the predetermined feature points based on the first location of that feature point corrected by said correction unit;
a second search unit configured to search the second search area for a second location of each of the predetermined feature points;
a determination unit configured to determine reliability of the second location of each of the predetermined feature points searched out of the second search area; and
a selection unit configured to select, as a final location for each of the predetermined feature points, either the first location corrected by said correction unit or the second location searched out of the second search area, based on the reliability of the second location, respectively.
14. A non-transitory computer-readable storage medium storing a program which makes a computer perform the method according to claim 1 .
15. The method according to claim 6 , wherein in said second search step, the search for the second location of each of the predetermined feature points is performed based on the reliability map.
16. The method according to claim 1 , further comprising:
a clipping step of clipping an object image out of an input image;
a normalization step of normalizing a size and/or an orientation of the object image;
wherein in said first search step, the search is performed on the normalized object image.
17. The method according to claim 16 , further comprising a preprocessing step of preprocessing an image by performing a color conversion or a contrast correction on the image and providing the preprocessed image as the input image for the clipping step.