Face tracking for controlling imaging parameters
View Patent ↗A method of tracking faces in an image stream with a digital image acquisition device includes receiving images from an image stream including faces, calculating corresponding integral images, and applying different subsets of face detection rectangles to the integral images to provide sets of candidate regions. The different subsets include candidate face regions of different sizes and/or locations within the images. The different candidate face regions from different images of the image stream are each tracked.
1. A face detection and recognition method, comprising:
acquiring, from an image stream, an image including one or more face regions;
calculating an integral image for at least a portion of the image or a sub-sampled version of the at least a portion of the image, or both;
applying a face detection to at least a portion of the integral image to provide one or more candidate face regions;
applying a face recognition, which uses a database, to the one or more candidate face regions to recognize a face in the one or more candidate face regions, and determining an identifier for the recognized face;
storing the identifier for the recognized face in association with the image or a portion thereof that includes the recognized face; and
adjusting one or more image acquisition conditions in an image acquiring device to modify white balance, color balance, focus, or exposure, or combinations thereof, for the recognized face.
2. The method of claim 1 , further comprising applying a face tracking to detect the recognized face from another image of the image stream.
3. The method of claim 1 , wherein the database includes an identifier and one or more associated parameters for each of one or more faces to be recognized.
4. The method of claim 1 , wherein the applying the face recognition is performed selectively, and the method further comprises selecting the one or more candidate face regions having a frontal alignment for the face recognition or including two eye regions and a mouth region within a given area of the candidate face region, or both.
5. The method of claim 1 , further comprising providing a level of confidence for each candidate face region based on the face detection, and wherein a candidate face region is designated with a first level of confidence as a face region, and wherein the face recognition is applied to the candidate face regions having a second confidence level higher than the first level of confidence.
6. The method of claim 1 , further comprising applying an Active Appearance Model (AAM) to the one or more candidate face regions, and applying the face recognition to the one or more candidate face regions having AAM parameters indicating the candidate face region that has a degree of rotation within a pre-determined range.
7. The method of claim 1 , further comprising applying chain classifiers to the image, wherein at least one of the chain classifiers includes a frontal classifier chain, and applying the face recognition to the one or more candidate face regions detected with the frontal classifier chain.
8. The method of claim 1 , further comprising calculating an average luminance for each candidate face region, and applying the face recognition to the one or more candidate face regions having the average luminance within a pre-determined range.
9. The method of claim 1 , wherein the image received from the image stream comprises a relatively low resolution preview image.
10. The method of claim 9 , further comprising determining a subsequent location or a subsequent size, or both, for the same face in a subsequent preview image; and based on an initial location or an initial size and the subsequent location or size, or combinations thereof, predicting a region of a third preview image which has just been acquired within which region the same face is expected to occur again.
11. A digital image acquisition device, comprising a lens, an image sensor, a processor, and a processor-readable memory having digital code embedded therein for programming the processor to perform a method of detecting and recognizing faces in digital images acquired by the device, wherein the method comprises:
acquiring, from an image stream, an image including one or more face regions;
calculating an integral image for at least a portion of the image or a sub-sampled version of the at least a portion of the image, or both;
applying a face detection to at least a portion of the integral image to provide one or more candidate face regions;
applying a face recognition, which uses a database, to the one or more candidate face regions to recognize a face in the one or more candidate face regions, and determining an identifier for the recognized face;
storing the identifier for the recognized face in association with the image or a portion thereof that includes the recognized face; and
adjusting one or more image acquisition conditions in an image acquiring device to modify white balance, color balance, focus, or exposure, or combinations thereof, for the recognized face.
12. The device of claim 11 , wherein the method further comprises applying a face tracking to detect the recognized face from another image of the image stream.
13. The device of claim 11 , wherein the database includes an identifier and one or more associated parameters for each of one or more faces to be recognized.
14. The device of claim 11 , wherein the applying the face recognition is performed selectively, and the method further comprises selecting the one or more candidate face regions having a frontal alignment for the face recognition or including two eye regions and a mouth region within a given area of the candidate face region, or both.
15. The device of claim 11 , further comprising providing a level of confidence for each candidate face region based on the face detection, and wherein a candidate face region is designated with a first level of confidence as a face region, and wherein the face recognition is applied to the candidate face regions having a second confidence level higher than the first level of confidence.
16. The device of claim 11 , further comprising applying an Active Appearance Model (AAM) to the one or more candidate face regions, and applying the face recognition to the one or more candidate face regions having AAM parameters indicating the candidate face region that has a degree of rotation within a pre-determined range.
17. The device of claim 11 , further comprising applying chain classifiers to the image, wherein at least one of the chain classifiers includes a frontal classifier chain, and applying the face recognition to the one or more candidate face regions detected with the frontal classifier chain.
18. The device of claim 11 , further comprising calculating an average luminance for each candidate face region, and applying the face recognition to the one or more candidate face regions having the average luminance within a pre-determined range.
19. The device of claim 11 , wherein the image received from the image stream comprises a relatively low resolution preview image.
20. The device of claim 19 , further comprising determining a subsequent location or a subsequent size, or both, for the same face in a subsequent preview image; and based on an initial location or an initial size and the subsequent location or size, or combinations thereof, predicting a region of a third preview image which has just been acquired within which region the same face is expected to occur again.
21. One or more non-transitory processor-readable storage media having code embedded therein for programming a processor to perform a method of detecting and recognizing faces in digital images received from an image stream that include one or more face regions, the method comprising:
calculating an integral image for at least a portion of a digital image or a sub-sampled version of the at least a portion of the image, or both;
applying a face detection to at least a portion of the integral image to provide one or more candidate face regions;
applying a face recognition, which uses a database, to the one or more candidate face regions to recognize a face in the one or more candidate face regions, and determining an identifier for the recognized face;
storing the identifier for the recognized face in association with the image or a portion thereof that includes the recognized face; and
adjusting one or more image acquisition conditions in an image acquiring device to modify white balance, color balance, focus, or exposure, or combinations thereof, for the recognized face.
22. The one or more processor-readable storage media of claim 21 , wherein the method further comprises applying a face tracking to detect the recognized face from another image of the image stream.
23. The one or more processor-readable storage media of claim 21 , wherein the database includes an identifier and one or more associated parameters for each of one or more faces to be recognized.
24. The one or more processor-readable storage media of claim 21 , wherein the applying the face recognition is performed selectively, and the method further comprises selecting the one or more candidate face regions having a frontal alignment for the face recognition or including two eye regions and a mouth region within a given area of the candidate face region, or both.
25. The one or more processor-readable storage media of claim 21 , further comprising providing a level of confidence for each candidate face region based on the face detection, and wherein a candidate face region is designated with a first level of confidence as a face region, and wherein the face recognition is applied to the candidate face regions having a second confidence level higher than the first level of confidence.
26. The one or more processor-readable storage media of claim 21 , further comprising applying an Active Appearance Model (AAM) to the one or more candidate face regions, and applying the face recognition to the one or more candidate face regions having AAM parameters indicating the candidate face region that has a degree of rotation within a pre-determined range.
27. The one or more processor-readable storage media of claim 21 , further comprising applying chain classifiers to the image, wherein at least one of the chain classifiers includes a frontal classifier chain, and applying face recognition to the one or more candidate face regions detected with the frontal classifier chain.
28. The one or more processor-readable storage media of claim 21 , further comprising calculating an average luminance for each candidate face region, and applying the face recognition to the one or more candidate face regions having the average luminance within a pre-determined range.
29. The one or more processor-readable storage media of claim 21 , wherein the image received from the image stream comprises a relatively low resolution preview image.
30. The one or more processor-readable storage media of claim 29 , further comprising determining a subsequent location or a subsequent size, or both, for the same face in a subsequent preview image; and based on based on an initial location or an initial size and the subsequent location or size, or combinations thereof, predicting a region of a third preview image which has just been acquired within which region the same face is expected to occur again.