IP Library Granted Patent US 9,224,034
Granted Patent B2
US 9,224,034 · App. 14/579,773 · Granted Dec 29, 2015

Face searching and detection in a digital image acquisition device

Inventors: Eran Steinberg (San Francisco, CA); Petronel Bigioi (Galway, IE); Peter Corcoran (Claregalway, IE); Mihnea Gangea (Bucharest, RO); Stefan Mirel Petrescu (Bucharest, RO); Andrei Vasiliu (Bucharest, RO); Gabriel Costache (Galway, IE); Alexandru Drimbarean (Galway, IE)
Assignee: FotoNation Limited
G06K9/00268G06K9/00221G06K9/00228G06K9/00234G06K9/00241
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,224,034
App. No.
14/579,773
Granted
Dec 29, 2015
Kind
B2
Abstract

A method of detecting a face in an image includes performing face detection within a first window of the image at a first location. A confidence level is obtained from the face detection indicating a probability of the image including a face at or in the vicinity of the first location. Face detection is then performed within a second window at a second location, wherein the second location is determined based on the confidence level.

Claims (67)

1. A device, comprising:

an image acquiring unit communicatively coupled to a memory unit, and configured to acquire data of a digital image that depicts one or more objects; and

a face detection unit configured to:

perform face detection within a first relatively large window of said image at a first location;

obtain from said face detection a confidence level that is below a threshold indicating a probability that no face is present at or in a vicinity of said first location;

perform second face detection within a second relatively large window of said image at a second location displaced from the first location by a first x-amount in a first direction and/or by a first y-amount in an orthogonal second direction;

obtain from said second face detection a confidence level that is at or above said threshold indicating a probability that the image at least may include a face in the vicinity of the second location;

apply a sequence of windows, including relatively smaller windows than said second relatively large window, at further locations in the vicinity of said second location, including a third location that is displaced from the second location by a second x-amount that is smaller than the first x-amount in the first direction and/or by a second y-amount that is smaller than the first y-amount in the orthogonal second direction wherein a magnitude of displacement by respective values of x-amount in a first direction and/or by respective values of y-amount in an orthogonal second direction is variable between successive windows of said sequence of windows within said image and wherein said x-amount and/or said y-amount decreases between successive windows when a confidence level indicating that the image at least may include a face increases across one or more further thresholds; and

repeat the obtaining of confidence levels until face detection for multiple predetermined sizes of windows has been performed over an entire region of interest of said image.

2. The device of claim 1 , wherein the face detection unit is further configured to:

repeat the obtaining of confidence levels below the threshold for multiple additional windows in different locations indicating probabilities in each case that no face is present at the different locations before said performing said second face detection.

3. The device of claim 1 , wherein the face detection unit is further configured to:

repeat the obtaining of confidence levels for multiple additional windows in different locations until face detection has been performed over an entire region of interest.

4. The device of claim 3 , wherein dimensions of said one or more additional windows depend on said confidence level.

5. The device of claim 1 , wherein the face detection unit is further configured to:

determine a set of regions of interest for said image, and

repeat the obtaining of confidence levels for each of all said regions of interest of said set.

6. The device of claim 1 , wherein the face detection unit is further configured to:

identify at least one region of said image likely to contain a face; and

repeat the obtaining of confidence levels until face detection is performed over substantially an entirety of said at least one region.

7. The device of claim 1 , wherein the face detection unit is further configured to:

identify at least one region of said image likely to contain a face; and

repeat the obtaining of confidence levels until face detection is performed concentrically inside said at least one region.

8. The device of claim 1 , wherein the face detection unit is further configured to: apply a chain of classifiers to said windows of said image and wherein said confidence level comprises a function of a number of classifiers successfully applied to said windows.

9. The device of claim 1 , wherein the first relatively large window of said image at a first location includes a portion that does not overlap the second window.

10. The device of claim 1 , wherein a size of the first window is decreased based on a confidence level of said face detection.

11. The device of claim 1 , wherein the magnitude of the second x-amount or the magnitude of the second y-amount, or both, are greater than the respective magnitudes of the first x-amount or the second y-amount, or both, for a higher confidence level that the image does not include a face at the first location compared with the second location.

12. The device of claim 1 , wherein an x-amount displacement magnitude and an y-amount displacement magnitude between at least two successive locations depend separately on face detection confidence levels in a first orthogonal direction and a second orthogonal direction, respectively.

13. The device of claim 1 , wherein the confidence level is above a threshold value that a face is detected in the vicinity of said first location.

14. The device of claim 1 , wherein the face detection unit is further configured to:

pre-determine one or more regions of interest within the image each as having an enhanced likelihood of including a face,

locate a detection window at a suitable corner of each such region of interest,

apply to each such region of interest a sequence of windows, by applying at said suitable corner a face detection window of relatively larger size than a subsequent face detection window that is applied upon moving a first displacement magnitude amount to a subsequent location upon obtaining a confidence level above at least one of said further thresholds for face detection at the previous location, and

displace a smaller magnitude of displacement distance from the previous location to the subsequent location when a level of confidence that a face may exist at the previous location exceeds said at least one of said further thresholds.

15. The device of claim 1 , wherein the face detection unit is further configured to: apply said sequence of windows to two or more regions of interest in time periods with at least some temporal overlap.

16. The device of claim 1 , wherein said one or more regions of interest comprise one or more regions including a number of skin pixels.

17. The device of claim 1 , wherein said image comprises is-an image in a stream of images and wherein said regions of interest comprise one or more regions in which a face has been detected in a previous image of said stream.

18. The device of claim 1 , wherein the face detection unit is further configured to:

responsive to said confidence level indicating a face at or in the vicinity of said first location, perform detection of a frontally aligned face within the first relatively large window.

19. The device of claim 1 , wherein the face detection unit is further configured to:

responsive to detection of a frontally aligned face in any candidate region, selectively apply face recognition to said candidate region.

20. The device of claim 1 , wherein the face detection unit is further configured to: apply face recognition in response to any candidate region having a probability of including a face greater than a predetermined threshold.

21. The device of claim 20 , wherein said performing detection is performed in a candidate region including said face detected at or in the vicinity of said first location.

22. The device of claim 1 , wherein said performing detection is performed in response to a candidate region having a probability of including a face greater than a first threshold.

23. The device of claim 1 , wherein the face detection unit is further configured to:

responsive to said first location having a probability of including a face less than a first threshold, extract an extended region including said first location from said image and perform face detection in said extended region with a relatively fine granularity to provide a candidate region.

24. The device of claim 1 , wherein the face detection unit is further configured to:

responsive to a candidate region having a probability of including a face less than a second threshold, rotate an extended region including a face region through one of a sequence of angles and perform face detection in said rotated extended region with a relatively fine granularity to provide another candidate region.

25. The device of claim 1 , wherein the face detection unit is further configured to:

responsive to a candidate region having a probability of including a face less than a second threshold, rotate said face within an extended region through one of a sequence of angles and perform face detection in said extended region with a relatively fine granularity to provide another candidate region.

26. A method for improving face searching and detection in a digital image acquisition device, the method comprising:

acquiring, using an image acquiring unit communicatively coupled to a memory unit, data of a digital image that depicts one or more objects;

using a face detection unit performing:

performing face detection within a first relatively large window of said image at a first location;

obtaining from said face detection a confidence level that is below a threshold indicating a probability that no face is present at or in a vicinity of said first location;

performing second face detection within a second relatively large window of said image at a second location displaced from the first location by a first x-amount in a first direction and/or by a first y-amount in an orthogonal second direction;

obtaining from said second face detection a confidence level that is at or above said threshold indicating a probability that the image at least may include a face in the vicinity of the second location;

applying a sequence of windows, including relatively smaller windows than said second relatively large window, at further locations in the vicinity of said second location, including a third location that is displaced from the second location by a second x-amount that is smaller than the first x-amount in the first direction and/or by a second y-amount that is smaller than the first y-amount in the orthogonal second direction wherein a magnitude of displacement by respective values of x-amount in a first direction and/or by respective values of y-amount in an orthogonal second direction is variable between successive windows of said sequence of windows within said image and wherein said x-amount and/or said y-amount decreases between successive windows when a confidence level indicating that the image at least may include a face increases across one or more further thresholds; and

repeating the obtaining of confidence levels until face detection for multiple predetermined sizes of windows has been performed over an entire region of interest of said image.

27. The method of claim 26 , further comprising:

repeating the obtaining of confidence levels below the threshold for multiple additional windows in different locations indicating probabilities in each case that no face is present at the different locations before said performing said second face detection.

28. The method of claim 26 , further comprising:

repeating the obtaining of confidence levels for multiple additional windows in different locations until face detection has been performed over an entire region of interest.

29. The method of claim 28 , wherein dimensions of said one or more additional windows depend on said confidence level.

30. The method of claim 26 , further comprising:

determining a set of regions of interest for said image, and

repeating the obtaining of confidence levels for each of all said regions of interest of said set.

Assignments (9)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 12, 2025
From: TOBII TECHNOLOGIES LTD
To: ADEIA MEDIA HOLDINGS LLC
Reel/Frame 071572/0855 →
CONVERSION Recorded Jun 12, 2025
From: ADEIA MEDIA HOLDINGS LLC
To: ADEIA MEDIA HOLDINGS INC.
Reel/Frame 071577/0875 →
SECURITY INTEREST Recorded May 28, 2025
From: ADEIA INC. (F/K/A XPERI HOLDING CORPORATION); ADEIA HOLDINGS INC.; ADEIA MEDIA HOLDINGS INC.; ADEIA IMAGING LLC; ADEIA MEDIA LLC; ADEIA MEDIA SOLUTIONS INC.; ADEIA SEMICONDUCTOR BONDING TECHNOLOGIES INC.; ADEIA TECHNOLOGIES INC.; ADEIA GUIDES INC.; ADEIA SOLUTIONS LLC; ADEIA SEMICONDUCTOR ADVANCED TECHNOLOGIES INC.; ADEIA SEMICONDUCTOR SOLUTIONS LLC; ADEIA SEMICONDUCTOR INTELLECTUAL PROPERTY LLC; ADEIA SEMICONDUCTOR TECHNOLOGIES LLC; ADEIA PUBLISHING INC.
To: BANK OF AMERICA, N.A., AS COLLATERAL AGENT
Reel/Frame 071454/0343 →
CHANGE OF NAME Recorded Mar 31, 2025
From: FOTONATION LIMITED
To: TOBII TECHNOLOGIES LIMITED
Reel/Frame 070679/0588 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 17, 2025
From: STEINBERG, ERAN; GANGEA, MIHNEA; PETRESCU, STEFAN; VASILIU, ANDREI; BIGIOI, PETRONEL; CORCORAN, PETER; COSTACHE, GABRIEL; DRIMBAREAN, ALEXANDRU
To: FOTONATION IRELAND LIMITED
Reel/Frame 070228/0730 →
CHANGE OF NAME Recorded Feb 17, 2025
From: FOTONATION LIMITED
To: TOBII TECHNOLOGY LIMITED
Reel/Frame 070239/0014 →
CHANGE OF NAME Recorded Feb 17, 2025
From: FOTONATION IRELAND LIMITED
To: TESSERA TECHNOLOGIES IRELAND LIMITED
Reel/Frame 070228/0762 →
CHANGE OF NAME Recorded Feb 17, 2025
From: TESSERA TECHNOLOGIES IRELAND LIMITED
To: DIGITALOPTICS CORPORATION EUROPE LIMITED
Reel/Frame 070228/0773 →
CHANGE OF NAME Recorded Feb 17, 2025
From: DIGITALOPTICS CORPORATION EUROPE LIMITED
To: FOTONATION LIMITED
Reel/Frame 070228/0794 →
Continuity (4)
Continuation 14177212 · Feb 10, 2014
Continuation 12374040
Provisional Application 60892883 · Mar 5, 2007
Related Publication 20150169941A1 · Jun 18, 2015