IP Library Granted Patent US 9,818,023
Granted Patent B2
US 9,818,023 · App. 15/415,888 · Granted Nov 14, 2017

Enhanced face detection using depth information

Inventors: Yael Shor (Tel Aviv, IL); Tomer Yanir (Rinnatya, IL); Yaniv Shaked (Binyamina, IL)
Assignee: APPLE INC.
G06K9/00241G06K9/00201
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,818,023
App. No.
15/415,888
Granted
Nov 14, 2017
Kind
B2
Abstract

A method for face detection includes capturing a depth map and an image of a scene and selecting one or more locations in the image to test for presence of human faces. At each selected location, a respective face detection window is defined, having a size that is scaled according to a depth coordinate of the location that is indicated by the depth map. Apart of the image that is contained within each face detection window is processed to determine whether the face detection window contains a human face. Similar methods may also be applied in identifying other object types.

Claims (27)

1. A method for face detection, comprising:

capturing a depth map, comprising an array of three-dimensional coordinates, and a two-dimensional image of a scene;

selecting one or more locations in the image to test for presence of human faces;

at each selected location, defining a respective face detection window having a size that is scaled according to an expected face size, in pixels of the image, at a depth of the selected location that is indicated by the depth map; and

processing a part of the image that is contained within each face detection window to determine whether the face detection window contains a human face.

2. The method according to claim 1 , wherein defining the respective face detection window comprises setting the size of the face detection window according to an extent, in pixels of the image, of an average face at the depth indicated by the depth map.

3. The method according to claim 1 , wherein processing the part of the image comprises normalizing the part of the image within each face detection window to a predetermined size, and applying a face detection algorithm to the normalized part of the image.

4. The method according to claim 3 , wherein applying the face detection algorithm comprises comparing the normalized part of the image within each face detection window to a single template, which is used to test for the presence of human faces at all of the selected locations.

5. The method according to claim 1 , further comprising calibrating the expected face size, using a reference image that is known to contain a face at a known depth.

6. The method according to claim 5 , wherein calibrating the expected face size comprises calibrating the expected face size by linearly scaling a size of the face in the reference image.

7. The method according to claim 1 , wherein, at each selected location, the face detection window having the size that is scaled according to the expected face size is the only face detection window defined in processing the image.

8. Apparatus for face detection, comprising:

an imaging assembly, which is configured to capture a depth map, comprising an array of three-dimensional coordinates, and a two-dimensional image of a scene; and

a processor, which is configured to select one or more locations in the image to test for presence of human faces, to define, at each selected location, a respective face detection window having a size that is scaled according to an expected face size, in pixels of the image, at a depth of the selected location that is indicated by the depth map, and to process a part of the image that is contained within each face detection window to determine whether the face detection window contains a human face.

9. The apparatus according to claim 8 , wherein the size of the face detection window is set according to an extent, in pixels of the image, of an average face at the depth indicated by the depth map.

10. The apparatus according to claim 8 , wherein the processor is configured to normalize the part of the image within each face detection window to a predetermined size, and to apply a face detection algorithm to the normalized part of the image.

11. The apparatus according to claim 10 , wherein the face detection algorithm comprises comparing the normalized part of the image within each face detection window to a single template, which is used to test for the presence of human faces at all of the selected locations.

12. The apparatus according to claim 8 , wherein the processor is further configured to calibrate the expected face size, using a reference image that is known to contain a face at a known depth.

13. The apparatus according to claim 12 , wherein the processor is configured to calibrate the expected face size by linearly scaling a size of the face in the reference image.

14. The apparatus according to claim 8 , wherein, at each selected location, the face detection window having the size that is scaled according to the expected face size is the only face detection window the processor is configured to define.

15. A method for image processing, comprising:

capturing a depth map, comprising an array of three-dimensional coordinates, and a two-dimensional image of a scene;

setting a base window size based on a characteristic size of a given object type that is to be identified in the scene;

selecting one or more locations in the image to test for presence of objects of the given type;

at each selected location, defining a respective detection window having a size that is scaled, relative to the base window size, according to an expected size of the given object type, in pixels of the image, at a depth of the location that is indicated by the depth map; and

processing a part of the image that is contained within each detection window to determine whether the detection window contains an object of the given type.

16. The method according to claim 15 , wherein processing the part of the image comprises detecting a selected part of a human body, and wherein the base window size is set according to an average size of the selected part.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 29, 2017
From: SHOR, YAEL; YANIR, TOMER; SHAKED, YANIV
To: PRIMESENSE LTD.
Reel/Frame 041114/0233 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 29, 2017
From: PRIMESENSE LTD.
To: APPLE INC.
Reel/Frame 041114/0239 →
Continuity (4)
Continuation 14664868 · Mar 22, 2015
Continuation 13751173 · Jan 28, 2013
Provisional Application 61595092 · Feb 5, 2012
Related Publication 20170132456A1 · May 11, 2017