Face tracking for controlling imaging parameters
A method of tracking faces in an image stream with a digital image acquisition device includes receiving images from an image stream including faces, calculating corresponding integral images, and applying different subsets of face detection rectangles to the integral images to provide sets of candidate regions. The different subsets include candidate face regions of different sizes and/or locations within the images. The different candidate face regions from different images of the image stream are each tracked.
1. A face detection and recognition method, comprising:
receiving a first preview image of a scene and a second preview image of nominally the same scene;
determining whether the first preview image comprises one or more face regions;
in response to determining that the first preview image comprises one or more face regions:
determining a first location of a first face region in the first preview image;
based at least in part on the first location of the first face region in the first preview image, predicting a second location in the second preview image of a second face region corresponding to the first face region in the first preview image;
based at least in part on pixel information of the second face region in the second preview image, determining one or more characteristics of the second face region in the second preview image;
based at least in part on the one or more characteristics of the second face region in the second preview image, determining one or more acquisition parameters for acquiring a main image of nominally the same scene; and
acquiring the main image of nominally the same scene using the one or more acquisition parameters;
wherein the method is performed using one or more computing devices.
2. The face detection and recognition method of claim 1 , further comprising:
in response to determining that the first preview image comprises the one or more face regions:
determining a first size of a third face region in the first preview image;
based at least in part on the first size of the third face region in the first preview image, predicting a second size in the second preview image of a fourth face region corresponding to the third face region in the first preview image;
based at least in part on additional information about the fourth face region in the second preview image, determining one or more additional characteristics of the fourth face region in the second preview image;
based at least in part on the one or more additional characteristics of the fourth face region in the second preview image, adjusting the one or more acquisition parameters for acquiring the main image of nominally the same scene; and
acquiring the main image of nominally the same scene using the adjusted one or more acquisition parameters.
3. The face detection and recognition method of claim 1 ,
wherein the one or more characteristics of the second region include any one of: sharpness, luminance, texture, color histogram, luminance histogram, horizontal luminance profile, vertical luminance profile, horizontal chrominance profile, vertical chrominance profile, or region correlogram;
wherein the one or more acquisition parameters for acquiring the main image include any one of: a white balance parameter, a color balance parameter, a focus parameter, or an exposure parameter; and
wherein the one or more acquisition parameters are used to perform any one of: auto-focus functions, auto-exposure functions, auto-white balance functions, or auto-color balance functions.
4. The face detection and recognition method of claim 1 , further comprising:
in response to determining the first location of the first face region in the first preview image:
determining, based on a face detection, a first level of confidence that the first face region is present in the first preview image;
determining a second level of confidence for predicting that the second face region, corresponding to the first face region in the first preview image, is present in the second preview image;
based on the first level of confidence and the second level of confidence, determining a cumulative confidence level; and
storing a history of the face detection in the first preview image, the cumulative confidence level, the first location of the first face region, and the second location of the second face region.
5. The face detection and recognition method of claim 1 ,
wherein the pixel information of the second face region includes any one of: color information of pixels includes in the second face region, or luminance information of the pixels included in the second face region.
6. The face detection and recognition method of claim 1 , wherein the predicted second face region in the second preview image of the second face region, corresponding to the first face region in the first preview image, is larger than the first face region in the first preview image.
7. The face detection and recognition method of claim 1 , wherein the determining whether the first preview image comprises one or more face regions comprises:
calculating an integral image for at least a portion of the first preview image;
applying a face detection to at least a portion of the integral image to provide one or more candidate face regions; and
applying one or more face recognition classifiers to the one or more candidate face regions to recognize one or more faces in the one or more candidate face regions.
8. A digital image acquisition device, comprising a lens, an image sensor, a processor, and one or more instructions which, when executed by the processor, cause the processor to perform:
receiving a first preview image of a scene and a second preview image of nominally the same scene;
determining whether the first preview image comprises one or more face regions;
in response to determining that the first preview image comprises one or more face regions:
determining a first location of a first face region in the first preview image;
based at least in part on the first location of the first face region in the first preview image, predicting a second location in the second preview image of a second face region corresponding to the first face region in the first preview image;
based at least in part on pixel information of the second face region in the second preview image, determining one or more characteristics of the second face region in the second preview image;
based at least in part on the one or more characteristics of the second face region in the second preview image, determining one or more acquisition parameters for acquiring a main image of nominally the same scene; and
acquiring the main image of nominally the same scene using the one or more acquisition parameters.
9. The digital image acquisition device of claim 8 , comprising additional instructions which, when executed by the processor, cause the processor to perform:
in response to determining that the first preview image comprises the one or more face regions:
determining a first size of a third face region in the first preview image;
based at least in part on the first size of the third face region in the first preview image, predicting a second size in the second preview image of a fourth face region corresponding to the third face region in the first preview image;
based at least in part on additional information about the fourth face region in the second preview image, determining one or more additional characteristics of the fourth face region in the second preview image;
based at least in part on the one or more additional characteristics of the fourth face region in the second preview image, adjusting the one or more acquisition parameters for acquiring the main image of nominally the same scene; and
acquiring the main image of nominally the same scene using the adjusted one or more acquisition parameters.
10. The digital image acquisition device of claim 8 ,
wherein the one or more characteristics of the second region include any one of: sharpness, luminance, texture, color histogram, luminance histogram, horizontal luminance profile, vertical luminance profile, horizontal chrominance profile, vertical chrominance profile, or region correlogram;
wherein the one or more acquisition parameters for acquiring the main image include any one of: a white balance parameter, a color balance parameter, a focus parameter, or an exposure parameter; and
wherein the one or more acquisition parameters are used to perform any one of: auto-focus functions, auto-exposure functions, auto-white balance functions, or auto-color balance functions.
11. The digital image acquisition device of claim 8 , comprising additional instructions which, when executed by the processor, cause the processor to perform:
in response to determining the first location of the first face region in the first preview image:
determining, based on a face detection, a first level of confidence that the first face region is present in the first preview image;
determining a second level of confidence for predicting that the second face region, corresponding to the first face region in the first preview image, is present in the second preview image;
based on the first level of confidence and the second level of confidence, determining a cumulative confidence level; and
storing a history of the face detection in the first preview image, the cumulative confidence level, the first location of the first face region, and the second location of the second face region.
12. The digital image acquisition device of claim 8 ,
wherein the pixel information of the second face region includes any one of: color information of pixels includes in the second face region, or luminance information of the pixels included in the second face region.
13. The digital image acquisition device of claim 8 ,
wherein the predicted second face region in the second preview image of the second face region, corresponding to the first face region in the first preview image, is larger than the first face region in the first preview image.
14. The digital image acquisition device of claim 8 , comprising additional instructions which, when executed by the processor, cause the processor to perform:
calculating an integral image for at least a portion of the first preview image;
applying a face detection to at least a portion of the integral image to provide one or more candidate face regions; and
applying one or more face recognition classifiers to the one or more candidate face regions to recognize one or more faces in the one or more candidate face regions.
15. A non-transitory computer readable storage medium storing one or more instructions which, when executed by one or more processors, cause the processors to perform:
receiving a first preview image of a scene and a second preview image of nominally the same scene;
determining whether the first preview image comprises one or more face regions;
in response to determining that the first preview image comprises one or more face regions:
determining a first location of a first face region in the first preview image;
based at least in part on the first location of the first face region in the first preview image, predicting a second location in the second preview image of a second face region corresponding to the first face region in the first preview image;
based at least in part on pixel information of the second face region in the second preview image, determining one or more characteristics of the second face region in the second preview image;
based at least in part on the one or more characteristics of the second face region in the second preview image, determining one or more acquisition parameters for acquiring a main image of nominally the same scene; and
acquiring the main image of nominally the same scene using the one or more acquisition parameters.
16. The non-transitory computer readable storage medium of claim 15 , comprising additional instructions which, when executed by the processor, cause the processor to perform:
in response to determining that the first preview image comprises the one or more face regions:
determining a first size of a third face region in the first preview image;
based at least in part on the first size of the third face region in the first preview image, predicting a second size in the second preview image of a fourth face region corresponding to the third face region in the first preview image;
based at least in part on additional information about the fourth face region in the second preview image, determining one or more additional characteristics of the fourth face region in the second preview image;
based at least in part on the one or more additional characteristics of the fourth face region in the second preview image, adjusting the one or more acquisition parameters for acquiring the main image of nominally the same scene; and
acquiring the main image of nominally the same scene using the adjusted one or more acquisition parameters.
17. The non-transitory computer readable storage medium of claim 15 ,
wherein the one or more characteristics of the second region include any one of: sharpness, luminance, texture, color histogram, luminance histogram, horizontal luminance profile, vertical luminance profile, horizontal chrominance profile, vertical chrominance profile, or region correlogram;
wherein the one or more acquisition parameters for acquiring the main image include any one of: a white balance parameter, a color balance parameter, a focus parameter, or an exposure parameter; and
wherein the one or more acquisition parameters are used to perform any one of: auto-focus functions, auto-exposure functions, auto-white balance functions, or auto-color balance functions.
18. The non-transitory computer readable storage medium of claim 15 , comprising additional instructions which, when executed by the processor, cause the processor to perform:
in response to determining the first location of the first face region in the first preview image:
determining, based on a face detection, a first level of confidence that the first face region is present in the first preview image;
determining a second level of confidence for predicting that the second face region, corresponding to the first face region in the first preview image, is present in the second preview image;
based on the first level of confidence and the second level of confidence, determining a cumulative confidence level; and
storing a history of the face detection in the first preview image, the cumulative confidence level, the first location of the first face region, and the second location of the second face region.
19. The non-transitory computer readable storage medium of claim 15 ,
wherein the pixel information of the second face region includes any one of: color information of pixels includes in the second face region, or luminance information of the pixels included in the second face region.
20. The non-transitory computer readable storage medium of claim 15 , comprising additional instructions which, when executed by the processor, cause the processor to perform:
calculating an integral image for at least a portion of the first preview image;
applying a face detection to at least a portion of the integral image to provide one or more candidate face regions; and
applying one or more face recognition classifiers to the one or more candidate face regions to recognize one or more faces in the one or more candidate face regions.