Detecting facial expressions in digital images
A method and system for detecting facial expressions in digital images and applications therefore are disclosed. Analysis of a digital image determines whether or not a smile and/or blink is present on a person's face. Face recognition, and/or a pose or illumination condition determination, permits application of a specific, relatively small classifier cascade.
1. A method of in-camera processing comprising:
acquiring a plurality of images;
identifying a plurality of faces in a first one of the plurality of images;
from among the plurality of faces, identifying a particular number of non-smiling faces by applying, to each identified face, a smile detection algorithm that has been trained with sample images of known smiling faces and sample images of known non-smiling faces as input; and
based, at least in part, on the particular number, initiating one or more operations, wherein the one or more operations include creating a composite image that includes a first region from at least one of the plurality of images and a second region from at least one other of the plurality of images.
2. The method of claim 1 wherein the one or more operations are initiated in response to the particular number being above a threshold number.
3. The method of claim 1 wherein:
the method further comprises determining that a particular percentage of the plurality of faces are smiling; and
the one or more operations are initiated in response to determining that the particular percentage is less than a threshold percentage.
4. The method of claim 1 wherein the one or more operations include issuing a warning beep.
5. The method of claim 1 wherein the at least one other of the plurality of images comprises a stream of preview images.
6. The method of claim 5 , wherein:
the method further comprises determining that a particular percentage of the plurality of faces are smiling,
the one or more operations are initiated in response to determining that the particular percentage is less than a threshold percentage; and
the first image is of higher resolution than the stream of preview images.
7. A portable digital image acquisition device, including a lens and an image sensor for acquiring digital images, a processor, and one or more processor-readable media comprising code for programming the processor to perform a method of in-camera processing, wherein the method comprises:
acquiring a plurality of images;
identifying a plurality of faces in a first one of the plurality of images;
from among the one or more faces, identifying a particular number of non-smiling faces by applying, to each identified face, a machine learning model trained to detect non-smiling faces; and
based, at least in part, on the particular number, initiating one or more operations, wherein the one or more operations include creating a composite image that includes a first region from at least one of the plurality of images and a second region from at least one other of the plurality of images.
8. The device of claim 7 wherein the one or more operations are initiated in response to the particular number being above a threshold number.
9. The device of claim 7 wherein:
the method further comprises determining that a particular percentage of the plurality of faces are smiling; and
the one or more operations are initiated in response to determining that the particular percentage is less than a threshold percentage.
10. The device of claim 7 wherein the one or more operations include issuing a warning beep.
11. The device of claim 7 wherein one or more of the plurality of images comprise a stream of preview images.
12. The device of claim 11 , wherein:
the method further comprises determining that a particular percentage of the plurality of faces are smiling,
the one or more operations are initiated in response to determining that the particular percentage is less than a threshold percentage; and
the first image is of higher resolution than the stream of preview images.
13. One or more non-transitory processor-readable media having code for programming a processor to perform a method of in-camera processing, wherein the method comprises:
acquiring a plurality of images;
identifying a plurality of faces in a first one of the plurality of images;
from among the plurality of faces, identifying a particular number of non-smiling faces by applying, to each identified face, a machine learning model trained to detect non-smiling faces; and
based, at least in part, on the particular number, initiating one or more operations, wherein the one or more operations include creating a composite image that includes a first region from at least one of the plurality of images and a second region from at least one other of the plurality of images.
14. The one or more processor-readable media of claim 13 wherein the one or more operations are initiated in response to the particular number being above a threshold number.
15. The one or more processor-readable media of claim 13 wherein:
the method further comprises determining that a particular percentage of the plurality of faces are smiling; and
the one or more operations are initiated in response to determining that the particular percentage is less than a threshold percentage.
16. The one or more processor-readable media of claim 13 wherein the one or more operations include issuing a warning beep.
17. The method of claim 2 wherein the first region comprises a first mouth region having a highest probability of being classified as a smile and the second region comprises a second mouth region having a highest probability of being classified as a smile.
18. The method of claim 3 wherein the first region comprises a first mouth region having a highest probability of being classified as a smile and the second region comprises a second mouth region having a highest probability of being classified as a smile.
19. The device of claim 8 wherein the first region comprises a first mouth region having a highest probability of being classified as a smile and the second region comprises a second mouth region having a highest probability of being classified as a smile.
20. The device of claim 9 wherein the first region comprises a first mouth region having a highest probability of being classified as a smile and the second region comprises a second mouth region having a highest probability of being classified as a smile.