Rapid auto-focus using classifier chains, MEMS and multiple object focusing
View Patent ↗A smart-focusing technique includes identifying an object of interest, such as a face, in a digital image. A focus-generic classifier chain is applied that is trained to match both focused and unfocused faces and/or data from a face tracking module is accepted. Multiple focus-specific classifier chains are applied, including a first chain trained to match substantially out of focus faces, and a second chain trained to match slightly out of focus faces. Focus position is rapidly adjusted using a MEMS component.
1. A method comprising:
acquiring data of a digital image which depicts one or more objects;
determining a plurality of in-focus probabilities by applying a plurality of in-focus classifier sets to the data;
wherein each in-focus probability indicates a likelihood that any of the one or more objects is depicted in focus in the digital image;
determining an in-focus cumulative probability based on the plurality of in-focus probabilities;
determining whether the in-focus cumulative probability is below an in-focus threshold;
in response to determining that the in-focus cumulative probability is below the in-focus threshold, performing the steps of:
determining a plurality of slightly-out-of-focus probabilities by applying a plurality of slightly-out-of-focus classifier sets to the data;
wherein each slightly-out-of-focus probability indicates a likelihood that any of the one or more objects is depicted slightly-out-of focus in the digital image;
determining slightly-out-of-focus cumulative probability based on the plurality of slightly-out-of-focus probabilities; and
in response to determining that the slightly-out-of-focus cumulative probability is equal to, or above, a slightly-out-of-focus threshold, determining a particular object, of the one or more objects, that is depicted in the digital image slightly-out-of-focus; and
wherein the method is performed by one or more computing devices.
2. The method of claim 1 ,
wherein an in-focus classifier set of the plurality of in-focus classifier sets comprises an in-focus-program configured to determine an in-focus probability that any of the one or more objects is depicted in-focus in the digital image;
wherein a slightly-out-of-focus classifier set of the one or more slightly-out-of-focus classifier sets comprises a slightly-out-of-focus-program configured to determine a slightly-out-of-focus probability that any of the one or more objects is depicted slightly-out-of-focus in the digital image; and
wherein the slightly-out-of-focus cumulative probability is computed from one or more slightly-out-of-focus probabilities determined by the one or more slightly-out-of-focus classifier sets.
3. The method of claim 2 , further comprising:
determining whether the slightly-out-of-focus cumulative probability is below a slightly-out-of-focus threshold; and
in response to determining that the slightly-out-of-focus cumulative probability is below, the slightly-out-of-focus threshold, applying one or more significantly-out-of-focus classifier sets to determine a significantly-out-of-focus cumulative probability.
4. The method of claim 3 ,
wherein a significantly-out-of-focus classifier set of the one or more significantly-out-of-focus classifier sets comprises a significantly-out-of-focus-program configured to determine a significantly-out-of-focus probability that any of the one or more objects is depicted significantly-out-of-focus in the digital image; and
wherein the significantly-out-of-focus cumulative probability is computed from one or more significantly-out-of-focus probabilities determined by the one or more significantly-out-of-focus classifier sets.
5. The method of claim 4 , further comprising:
determining whether the significantly-out-of-focus cumulative probability is equal to, or above, a significantly-out-of-focus threshold; and
in response to determining that the significantly-out-of-focus cumulative probability is equal to, or above, a significantly-out-of-focus threshold, determining the particular object, of the one or more objects, that is depicted in the digital image significantly-out-of-focus.
6. The method of claim 5 , further comprising:
determining whether the significantly-out-of-focus cumulative probability is below the significantly-out-of-focus threshold; and
in response to determining that the significantly-out-of-focus cumulative probability is below the significantly-out-of-focus threshold, determining that none of the one or more objects is depicted in the digital image.
7. The method of claim 6 , further comprising:
in response to determining that the in-focus cumulative probability is above, or equal to, the in-focus threshold, determining the particular object, of the one or more objects, that is depicted in-focus face in the digital image.
8. A non-transitory computer-readable storage medium, storing one or more computer instructions which, when executed by one or more processors, cause the one or more processors to perform:
acquiring data of a digital image which depicts one or more objects;
determining a plurality of in-focus probabilities by applying a plurality of in-focus classifier sets to the data;
wherein each in-focus probability indicates a likelihood that any of the one or more objects is depicted in focus in the digital image;
determining an in-focus cumulative probability based on the plurality of in-focus probabilities;
determining whether the in-focus cumulative probability is below an in-focus threshold;
in response to determining that the in-focus cumulative probability is below the in-focus threshold, performing the steps of:
determining a plurality of slightly-out-of-focus probabilities by applying a plurality of slightly-out-of-focus classifier sets to the data;
wherein each slightly-out-of-focus probability indicates a likelihood that any of the one or more objects is depicted slightly-out-of focus in the digital image;
determining slightly-out-of-focus cumulative probability based on the plurality of slightly-out-of-focus probabilities; and
in response to determining that the slightly-out-of-focus cumulative probability is equal to, or above, a slightly-out-of-focus threshold, determining a particular object, of the one or more objects, that is depicted in the digital image slightly-out-of-focus.
9. The non-transitory computer-readable storage medium of claim 8 ,
wherein an in-focus classifier set of the plurality of in-focus classifier sets comprises an in-focus-program configured to determine an in-focus probability that any of the one or more objects is depicted in-focus in the digital image;
wherein a slightly-out-of-focus classifier set of the one or more slightly-out-of-focus classifier sets comprises a slightly-out-of-focus-program configured to determine a slightly-out-of-focus probability that any of the one or more objects is depicted slightly-out-of-focus in the digital image; and
wherein the slightly-out-of-focus cumulative probability is computed from one or more slightly-out-of-focus probabilities determined by the one or more slightly-out-of-focus classifier sets.
10. The non-transitory computer-readable storage medium of claim 9 , storing additional instructions which, when executed, cause the one or more processors to perform:
determining whether the slightly-out-of-focus cumulative probability is below a slightly-out-of-focus threshold; and
in response to determining that the slightly-out-of-focus cumulative probability is below, the slightly-out-of-focus threshold, applying one or more significantly-out-of-focus classifier sets to determine a significantly-out-of-focus cumulative probability.
11. The non-transitory computer-readable storage medium of claim 10 ,
wherein a significantly-out-of-focus classifier set of the one or more significantly-out-of-focus classifier sets comprises a significantly-out-of-focus-program configured to determine a significantly-out-of-focus probability that any of the one or more objects is depicted significantly-out-of-focus in the digital image; and
wherein the significantly-out-of-focus cumulative probability is computed from one or more significantly-out-of-focus probabilities determined by the one or more significantly-out-of-focus classifier sets.
12. The non-transitory computer-readable storage medium of claim 11 , storing additional instructions which, when executed, cause the one or more processors to perform:
determining whether the significantly-out-of-focus cumulative probability is equal to, or above, a significantly-out-of-focus threshold; and
in response to determining that the significantly-out-of-focus cumulative probability is equal to, or above, a significantly-out-of-focus threshold, determining the particular object, of the one or more objects, that is depicted in the digital image significantly-out-of-focus.
13. The non-transitory computer-readable storage medium of claim 12 , storing additional instructions which, when executed, cause the one or more processors to perform:
determining whether the significantly-out-of-focus cumulative probability is below the significantly-out-of-focus threshold; and
in response to determining that the significantly-out-of-focus cumulative probability is below the significantly-out-of-focus threshold, determining that none of the one or more objects is depicted in the digital image.
14. The non-transitory computer-readable storage medium of claim 13 , storing additional instructions which, when executed, cause the one or more processors to perform:
in response to determining that the in-focus cumulative probability is above, or equal to, the in-focus threshold, determining the particular object, of the one or more objects, that is depicted in-focus face in the digital image.
15. A device, comprising:
an image retrieval unit coupled to one or more memory units and configured to acquire data of a digital image which depicts one or more objects; and
an image analysis unit configured to:
determine a plurality of in-focus probabilities by applying a plurality of in-focus classifier sets to the data;
wherein each in-focus probability indicates a likelihood that any of the one or more objects is depicted in focus in the digital image;
determine an in-focus cumulative probability based on the plurality of in-focus probabilities;
determine whether the in-focus cumulative probability is below an in-focus threshold;
in response to determining that the in-focus cumulative probability is below the in-focus threshold, performing the steps of:
determine a plurality of slightly-out-of-focus probabilities by applying a plurality of slightly-out-of-focus classifier sets to the data;
wherein each slightly-out-of-focus probability indicates a likelihood that any of the one or more objects is depicted slightly-out-of focus in the digital image;
determine slightly-out-of-focus cumulative probability based on the plurality of slightly-out-of-focus probabilities; and
in response to determining that the slightly-out-of-focus cumulative probability is equal to, or above, a slightly-out-of-focus threshold, determine a particular object, of the one or more objects, that is depicted in the digital image slightly-out-of-focus.
16. The device of claim 15 ,
wherein an in-focus classifier set of the plurality of in-focus classifier sets comprises an in-focus-program configured to determine an in-focus probability that any of the one or more objects is depicted in-focus in the digital image;
wherein a slightly-out-of-focus classifier set of the one or more slightly-out-of-focus classifier sets comprises a slightly-out-of-focus-program configured to determine a slightly-out-of-focus probability that any of the one or more objects is depicted slightly-out-of-focus in the digital image; and
wherein the slightly-out-of-focus cumulative probability is computed from one or more slightly-out-of-focus probabilities determined by the one or more slightly-out-of-focus classifier sets.
17. The device of claim 16 , wherein the image analysis unit is further configured to:
determine whether the slightly-out-of-focus cumulative probability is below a slightly-out-of-focus threshold; and
in response to determining that the slightly-out-of-focus cumulative probability is below, the slightly-out-of-focus threshold, apply one or more significantly-out-of-focus classifier sets to determine a significantly-out-of-focus cumulative probability.
18. The device of claim 17 ,
wherein a significantly-out-of-focus classifier set of the one or more significantly-out-of-focus classifier sets comprises a significantly-out-of-focus-program configured to determine a significantly-out-of-focus probability that any of the one or more objects is depicted significantly-out-of-focus in the digital image; and
wherein the significantly-out-of-focus cumulative probability is computed from one or snore significantly-out-of-focus probabilities determined by the one or more significantly-out-of-focus classifier sets.
19. The device of claim 18 , wherein the image analysis unit is further configured to:
determine whether the significantly-out-of-focus cumulative probability is equal to, or above, a significantly-out-of-focus threshold; and
in response to determining that the significantly-out-of-focus cumulative probability is equal to, or above, a significantly-out-of-focus threshold, determine the particular object, of the one or more objects, that is depicted in the digital image significantly-out-of-focus.
20. The device of claim 19 , wherein the image analysis unit is further configured to:
determine whether the significantly-out-of-focus cumulative probability is below the significantly-out-of-focus threshold; and
in response to determining that the significantly-out-of-focus cumulative probability is below the significantly-out-of-focus threshold, determine that none of the one or more objects is depicted in the digital image.