Method for locating faces in digital color images

1. A digital image processing method for locating faces in a digital color image, comprising the steps of:

generating an integral image from the digital color image;

generating a grid pattern image from said integral image, said grid pattern image having a grid pattern having a plurality of cells each of said cells having a plurality of pixels;

reducing said grid pattern image to a corresponding grid pattern element image, said grid pattern element image having a number of pixels equal to the number of cells in said grid pattern image; and

performing a correlation test between said grid pattern element image and a predetermined mean grid pattern element image of sample face images.

2. The method claimed in claim 1 wherein said generating an integral image step further comprises:

cropping a sub-image from said digital color image; and

computing said integral image from said sub-image.

3. The method claimed in claim 2 wherein said grid pattern has a first cell size and the method further comprises iterating said generating of said grid pattern image, said reducing, and said performing steps with each of a plurality of additional grid patterns, said grid patterns all having different cell sizes.

4. The method claimed in claim 3 wherein said digital color image has a plurality of different areas, said sub-image is of one of said areas; and the method further comprises repeating said generating, generating, reducing, performing, and iterating steps on a plurality of different additional sub-images of said digital color image.

5. The method claimed in claim 1 , wherein the step of generating the integral image further comprises the steps of:

replacing non-skin-color pixels with black to produce an image C having skin color pixels;

replacing non face shaped clusters with black to produce an image E having skin colored face shaped clusters;

labeling clusters of skin colored face shaped clusters; and

generating the integral image from each cluster of the image E.

6. The method claimed in claim 5 , wherein the step of replacing non face shaped clusters comprises the steps of:

clustering skin-color pixels in image C into clusters;

applying morphological opening and closing processes to skin-colored pixel clusters; and

replacing the pixels of a cluster with black if it does not meet a geometrical criterion for a face, resulting in an image E; and wherein the step of labeling skin colored clusters comprises the step of generating a linked list of sets of parameters including a starting position, width, and height, that defines regions containing a cluster of skin-colored pixels.

7. The method as claimed in claim 5 , wherein the step of selecting an irregular grid pattern comprises determining a plurality of different size grid cells that cover major features including eyes, nose, mouth, forehead, and cheek of the mean face image.

8. The method claimed in claim 1 further comprising prior to said generating steps:

generating a mean face image from a collection of sample face images;

generating a mean integral image from the mean face image;

generating a mean grid pattern image from said mean integral image, said mean grid pattern image having a plurality of cells in said grid pattern; and

reducing said mean grid pattern image to a corresponding mean grid pattern element image, said mean grid pattern element image having a number of pixels equal to the number of cells in said mean grid pattern image.

9. The method claimed in claim 8 wherein said grid pattern element images and said mean grid pattern element image are one-dimensional.

10. The method claimed in claim 8 , wherein the grid pattern is regular.

11. The method, claimed in claim 10 , wherein the step of selecting a regular grid pattern comprises:

computing a distance, e 1 , between two eye centers of the mean face image;

computing a center position, c, between two eye centers; and

using e 1 and c to determine a region (position and size) that contains M by N grid cells with each cell having m by n pixels.

12. The method claimed in claim 8 , wherein the grid pattern is irregular.

13. The method claimed in claim 8 , wherein the step of generating the mean integral image further comprises the steps of:

replacing non-skin-color pixels with black to produce an image C having skin color pixels;

replacing non face shaped clusters with black to produce an image E having skin colored face shaped clusters;

labeling clusters of skin colored face shaped clusters; and

generating the mean integral image from each cluster of the image E.

14. The method claimed in claim 13 , further comprising the steps of:

eliminating faces that contain more than a predetermined percentage of black pixels; and

merging faces that substantially overlap.

15. The method claimed in claim 13 , wherein the step of replacing non face shaped clusters comprises the steps of:

clustering skin-color pixels in image C into clusters;

applying morphological opening and closing processes to skin-colored pixel clusters; and

replacing the pixels of a cluster with black if it does not meet a geometrical criterion for a face, resulting in an image E; and wherein the step of labeling skin colored clusters comprises the step of generating a linked list of sets of parameters including a starting position, width, and height, that defines regions containing a cluster of skin-colored pixels.

16. The method claimed in claim 1 , wherein the grid pattern is regular.

17. The method claimed in claim 1 , wherein the grid pattern is irregular.

18. A computer program product for performing the method of claim 1 , the computer program product comprising computer readable storage medium having a computer program stored thereon.

19. A digital image processing method for locating faces in a digital color image, comprising the steps of:

a) generating a mean arid pattern element (MGPe) image from a plurality of sample face images;

b) generating an integral image from the digital color image; and

c) locating faces in the digital color image by using the integral image to perform a correlation test between the mean grid pattern element (MGPe) image and the digital color image at a plurality of effective resolutions by reducing the digital color image to a plurality of arid pattern element images (GPes) at different effective resolutions and correlating the MGPe with the GPes;

wherein the step of generating a mean arid pattern element (MGPe) image, comprises the steps of:

a 1 ) collecting sample face images;

a 2 ) generating a mean face image from the sample face images;

a 3 ) selecting a arid pattern (GP); and

a 4 ) reducing the resolution of the mean face image to the resolution of the selected arid pattern (GP) by averaging;

wherein the arid pattern is regular; and

wherein the step of selecting a regular grid pattern comprises computing a distance, e 1 , between two eye centers of the mean face image; computing a center position, c, between two eye centers; using e 1 and c to determine M and N dimensions and a position of a region wherein said region contains M by N grid cells with each cell having m by n pixels.

20. A digital image processing method for locating faces in a digital color image, comprising the steps of:

a) generating a mean grid pattern element (MGPe) image from a plurality of sample face images;

b) generating an integral image from the digital color image; and

c) locating faces in the digital color image by using the integral image to perform a correlation test between the mean grid pattern element (MGPe) image and the digital color image at a plurality of effective resolutions by reducing the digital color image to a plurality of grid pattern element images (GPes) at different effective resolutions and correlating the MGPe with the GPes;

wherein the step of generating an integral image further comprises the steps of:

b 1 ) replacing non-skin-color pixels with black to produce an image C having skin color pixels;

b 2 ) replacing non face shaped clusters with black to produce an image E having skin colored face shaped clusters;

b 3 ) labeling clusters of skin colored face shaped clusters; and

b 4 ) generating the integral image from each cluster of the image E.

21. The method claimed in claim 20 , further comprising the steps of:

d) eliminating faces that contain more than a predetermined percentage of black pixels; and

e) merging faces that substantially overlap.

22. The method claimed in claim 20 , wherein the step of replacing non face shaped clusters comprises the steps of:

b 2 . 1 ) clustering skin-color pixels in image C into clusters;

b 2 . 2 ) applying morphological opening and closing processes to skin-colored pixel clusters; and

b 2 . 3 ) replacing the pixels of a cluster with black if it does not meet a geometrical criterion for a face, resulting in an image E; and wherein the step of labeling skin colored clusters comprises the step of generating a linked list of sets of parameters including a starting position, width, and height, that defines regions containing a cluster of skin-colored pixels.

23. A digital image processing system for locating faces in a digital color image comprising:

means for generating an integral image from the digital color image;

means for generating a grid pattern image from said integral image, said grid pattern image having a grid pattern having a plurality of cells;

means for reducing said grid pattern image to a corresponding grid pattern element image, said grid pattern element image having a number of pixels equal to the number of cells in said grid pattern image; and

means for performing a correlation test between said grid pattern element image and a predetermined mean grid pattern element image of sample face images.