IP Library Granted Patent US 8,797,448
Granted Patent B2
US 8,797,448 · App. 14/174,821 · Granted Aug 5, 2014

Rapid auto-focus using classifier chains, MEMS and multiple object focusing

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,797,448
App. No.
14/174,821
Filed
Feb 6, 2014
Granted
Aug 5, 2014
Kind
B2
Examiner
TRAN, NHAN T
Art Unit
2664
USPC
348/345
Abstract

A smart-focusing technique includes identifying an object of interest, such as a face, in a digital image. A focus-generic classifier chain is applied that is trained to match both focused and unfocused faces and/or data from a face tracking module is accepted. Multiple focus-specific classifier chains are applied, including a first chain trained to match substantially out of focus faces, and a second chain trained to match slightly out of focus faces. Focus position is rapidly adjusted using a MEMS component.

Claims (87)

1. A method comprising:

acquiring data of a digital image which depicts one or more objects;

determining a plurality of in-focus probabilities by applying a plurality of in-focus classifier sets to the data;

wherein each in-focus probability indicates a likelihood that any of the one or more objects is depicted in focus in the digital image;

determining an in-focus cumulative probability based on the plurality of in-focus probabilities;

determining whether the in-focus cumulative probability is below an in-focus threshold;

in response to determining that the in-focus cumulative probability is below the in-focus threshold, performing the steps of:

determining a plurality of slightly-out-of-focus probabilities by applying a plurality of slightly-out-of-focus classifier sets to the data;

wherein each slightly-out-of-focus probability indicates a likelihood that any of the one or more objects is depicted slightly-out-of focus in the digital image;

determining slightly-out-of-focus cumulative probability based on the plurality of slightly-out-of-focus probabilities; and

in response to determining that the slightly-out-of-focus cumulative probability is equal to, or above, a slightly-out-of-focus threshold, determining a particular object, of the one or more objects, that is depicted in the digital image slightly-out-of-focus; and

wherein the method is performed by one or more computing devices.

2. The method of claim 1 ,

wherein an in-focus classifier set of the plurality of in-focus classifier sets comprises an in-focus-program configured to determine an in-focus probability that any of the one or more objects is depicted in-focus in the digital image;

wherein a slightly-out-of-focus classifier set of the one or more slightly-out-of-focus classifier sets comprises a slightly-out-of-focus-program configured to determine a slightly-out-of-focus probability that any of the one or more objects is depicted slightly-out-of-focus in the digital image; and

wherein the slightly-out-of-focus cumulative probability is computed from one or more slightly-out-of-focus probabilities determined by the one or more slightly-out-of-focus classifier sets.

3. The method of claim 2 , further comprising:

determining whether the slightly-out-of-focus cumulative probability is below a slightly-out-of-focus threshold; and

in response to determining that the slightly-out-of-focus cumulative probability is below, the slightly-out-of-focus threshold, applying one or more significantly-out-of-focus classifier sets to determine a significantly-out-of-focus cumulative probability.

4. The method of claim 3 ,

wherein a significantly-out-of-focus classifier set of the one or more significantly-out-of-focus classifier sets comprises a significantly-out-of-focus-program configured to determine a significantly-out-of-focus probability that any of the one or more objects is depicted significantly-out-of-focus in the digital image; and

wherein the significantly-out-of-focus cumulative probability is computed from one or more significantly-out-of-focus probabilities determined by the one or more significantly-out-of-focus classifier sets.

5. The method of claim 4 , further comprising:

determining whether the significantly-out-of-focus cumulative probability is equal to, or above, a significantly-out-of-focus threshold; and

in response to determining that the significantly-out-of-focus cumulative probability is equal to, or above, a significantly-out-of-focus threshold, determining the particular object, of the one or more objects, that is depicted in the digital image significantly-out-of-focus.

6. The method of claim 5 , further comprising:

determining whether the significantly-out-of-focus cumulative probability is below the significantly-out-of-focus threshold; and

in response to determining that the significantly-out-of-focus cumulative probability is below the significantly-out-of-focus threshold, determining that none of the one or more objects is depicted in the digital image.

7. The method of claim 6 , further comprising:

in response to determining that the in-focus cumulative probability is above, or equal to, the in-focus threshold, determining the particular object, of the one or more objects, that is depicted in-focus face in the digital image.

8. A non-transitory computer-readable storage medium, storing one or more computer instructions which, when executed by one or more processors, cause the one or more processors to perform:

acquiring data of a digital image which depicts one or more objects;

determining a plurality of in-focus probabilities by applying a plurality of in-focus classifier sets to the data;

wherein each in-focus probability indicates a likelihood that any of the one or more objects is depicted in focus in the digital image;

determining an in-focus cumulative probability based on the plurality of in-focus probabilities;

determining whether the in-focus cumulative probability is below an in-focus threshold;

in response to determining that the in-focus cumulative probability is below the in-focus threshold, performing the steps of:

determining a plurality of slightly-out-of-focus probabilities by applying a plurality of slightly-out-of-focus classifier sets to the data;

wherein each slightly-out-of-focus probability indicates a likelihood that any of the one or more objects is depicted slightly-out-of focus in the digital image;

determining slightly-out-of-focus cumulative probability based on the plurality of slightly-out-of-focus probabilities; and

in response to determining that the slightly-out-of-focus cumulative probability is equal to, or above, a slightly-out-of-focus threshold, determining a particular object, of the one or more objects, that is depicted in the digital image slightly-out-of-focus.

9. The non-transitory computer-readable storage medium of claim 8 ,

wherein an in-focus classifier set of the plurality of in-focus classifier sets comprises an in-focus-program configured to determine an in-focus probability that any of the one or more objects is depicted in-focus in the digital image;

wherein a slightly-out-of-focus classifier set of the one or more slightly-out-of-focus classifier sets comprises a slightly-out-of-focus-program configured to determine a slightly-out-of-focus probability that any of the one or more objects is depicted slightly-out-of-focus in the digital image; and

wherein the slightly-out-of-focus cumulative probability is computed from one or more slightly-out-of-focus probabilities determined by the one or more slightly-out-of-focus classifier sets.

10. The non-transitory computer-readable storage medium of claim 9 , storing additional instructions which, when executed, cause the one or more processors to perform:

determining whether the slightly-out-of-focus cumulative probability is below a slightly-out-of-focus threshold; and

in response to determining that the slightly-out-of-focus cumulative probability is below, the slightly-out-of-focus threshold, applying one or more significantly-out-of-focus classifier sets to determine a significantly-out-of-focus cumulative probability.

11. The non-transitory computer-readable storage medium of claim 10 ,

wherein a significantly-out-of-focus classifier set of the one or more significantly-out-of-focus classifier sets comprises a significantly-out-of-focus-program configured to determine a significantly-out-of-focus probability that any of the one or more objects is depicted significantly-out-of-focus in the digital image; and

wherein the significantly-out-of-focus cumulative probability is computed from one or more significantly-out-of-focus probabilities determined by the one or more significantly-out-of-focus classifier sets.

12. The non-transitory computer-readable storage medium of claim 11 , storing additional instructions which, when executed, cause the one or more processors to perform:

determining whether the significantly-out-of-focus cumulative probability is equal to, or above, a significantly-out-of-focus threshold; and

in response to determining that the significantly-out-of-focus cumulative probability is equal to, or above, a significantly-out-of-focus threshold, determining the particular object, of the one or more objects, that is depicted in the digital image significantly-out-of-focus.

13. The non-transitory computer-readable storage medium of claim 12 , storing additional instructions which, when executed, cause the one or more processors to perform:

determining whether the significantly-out-of-focus cumulative probability is below the significantly-out-of-focus threshold; and

in response to determining that the significantly-out-of-focus cumulative probability is below the significantly-out-of-focus threshold, determining that none of the one or more objects is depicted in the digital image.

14. The non-transitory computer-readable storage medium of claim 13 , storing additional instructions which, when executed, cause the one or more processors to perform:

in response to determining that the in-focus cumulative probability is above, or equal to, the in-focus threshold, determining the particular object, of the one or more objects, that is depicted in-focus face in the digital image.

15. A device, comprising:

an image retrieval unit coupled to one or more memory units and configured to acquire data of a digital image which depicts one or more objects; and

an image analysis unit configured to:

determine a plurality of in-focus probabilities by applying a plurality of in-focus classifier sets to the data;

wherein each in-focus probability indicates a likelihood that any of the one or more objects is depicted in focus in the digital image;

determine an in-focus cumulative probability based on the plurality of in-focus probabilities;

determine whether the in-focus cumulative probability is below an in-focus threshold;

in response to determining that the in-focus cumulative probability is below the in-focus threshold, performing the steps of:

determine a plurality of slightly-out-of-focus probabilities by applying a plurality of slightly-out-of-focus classifier sets to the data;

wherein each slightly-out-of-focus probability indicates a likelihood that any of the one or more objects is depicted slightly-out-of focus in the digital image;

determine slightly-out-of-focus cumulative probability based on the plurality of slightly-out-of-focus probabilities; and

in response to determining that the slightly-out-of-focus cumulative probability is equal to, or above, a slightly-out-of-focus threshold, determine a particular object, of the one or more objects, that is depicted in the digital image slightly-out-of-focus.

16. The device of claim 15 ,

wherein an in-focus classifier set of the plurality of in-focus classifier sets comprises an in-focus-program configured to determine an in-focus probability that any of the one or more objects is depicted in-focus in the digital image;

wherein a slightly-out-of-focus classifier set of the one or more slightly-out-of-focus classifier sets comprises a slightly-out-of-focus-program configured to determine a slightly-out-of-focus probability that any of the one or more objects is depicted slightly-out-of-focus in the digital image; and

wherein the slightly-out-of-focus cumulative probability is computed from one or more slightly-out-of-focus probabilities determined by the one or more slightly-out-of-focus classifier sets.

17. The device of claim 16 , wherein the image analysis unit is further configured to:

determine whether the slightly-out-of-focus cumulative probability is below a slightly-out-of-focus threshold; and

in response to determining that the slightly-out-of-focus cumulative probability is below, the slightly-out-of-focus threshold, apply one or more significantly-out-of-focus classifier sets to determine a significantly-out-of-focus cumulative probability.

18. The device of claim 17 ,

wherein a significantly-out-of-focus classifier set of the one or more significantly-out-of-focus classifier sets comprises a significantly-out-of-focus-program configured to determine a significantly-out-of-focus probability that any of the one or more objects is depicted significantly-out-of-focus in the digital image; and

wherein the significantly-out-of-focus cumulative probability is computed from one or snore significantly-out-of-focus probabilities determined by the one or more significantly-out-of-focus classifier sets.

19. The device of claim 18 , wherein the image analysis unit is further configured to:

determine whether the significantly-out-of-focus cumulative probability is equal to, or above, a significantly-out-of-focus threshold; and

in response to determining that the significantly-out-of-focus cumulative probability is equal to, or above, a significantly-out-of-focus threshold, determine the particular object, of the one or more objects, that is depicted in the digital image significantly-out-of-focus.

20. The device of claim 19 , wherein the image analysis unit is further configured to:

determine whether the significantly-out-of-focus cumulative probability is below the significantly-out-of-focus threshold; and

in response to determining that the significantly-out-of-focus cumulative probability is below the significantly-out-of-focus threshold, determine that none of the one or more objects is depicted in the digital image.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 19, 2025
From: FOTONATION LIMITED
To: ADEIA IMAGING LLC
Reel/Frame 073635/0320 →
SECURITY INTEREST Recorded May 3, 2023
From: ADEIA GUIDES INC.; ADEIA IMAGING LLC; ADEIA MEDIA HOLDINGS LLC; ADEIA MEDIA SOLUTIONS INC.; ADEIA SEMICONDUCTOR ADVANCED TECHNOLOGIES INC.; ADEIA SEMICONDUCTOR BONDING TECHNOLOGIES INC.; ADEIA SEMICONDUCTOR INC.; ADEIA SEMICONDUCTOR SOLUTIONS LLC; ADEIA SEMICONDUCTOR TECHNOLOGIES LLC; ADEIA SOLUTIONS LLC
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 063529/0272 →