IP Library Granted Patent US 10,785,393
Granted Patent B2
US 10,785,393 · App. 14/720,390 · Granted Sep 22, 2020

Methods and devices for selective flash illumination

Inventor: Shizhe Shen (San Mateo, CA)
Assignee: FACEBOOK, INC.
H04N5/2256G06K9/00201G06K9/2027G06K9/3241G06T7/50H04N5/2354H04N5/23216H04N5/23241G06K2209/401
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,785,393
App. No.
14/720,390
Granted
Sep 22, 2020
Kind
B2
Abstract

The various embodiments described herein include methods and/or devices for directing flash illumination. In one aspect, a method is performed at a device including a camera, one or more processors, and memory storing one or more programs configured for execution by the one or more processors. The method includes directing flash illumination to an identified area of interest in a scene by adjusting power supplied to respective elements of a flash array. The method further includes capturing, via the camera, an image of the scene as illuminated by the flash illumination.

Claims (50)

1. A method comprising:

at a device comprising a camera, one or more processors, and memory storing one or more programs configured for execution by the one or more processors:

constructing a depth map of an area of interest in a scene according to one or more images captured while applying a structured light pattern to the area of interest, including:

applying multiple, iterative structured light patterns to the area of interest to obtain multiple respective depth maps, wherein:

the area of interest in the scene comprises an object;

applying the structured light pattern comprises applying one or more lines of the structured light pattern with a specified orientation across an edge of the object so that a subset of the one or more lines of the structured light pattern intersect the edge of the object from at least one side at a specified angle; and

the structured light pattern is rotated for each iteration until the lines of the structured light pattern are determined to be perpendicular to the edge of the object;

directing, by the device, flash illumination to the area of interest in the scene, the directing comprising adjusting power supplied to respective elements of a flash array;

capturing, via the camera, an image of the scene as illuminated by the flash illumination;

predicting, by the device, subsequent movement of the object;

redirecting, by the device, flash illumination to the object based on the predicted movement of the object; and

capturing, via the camera, one or more additional images as the flash illumination is redirected while the object moves.

2. The method of claim 1 , wherein directing the flash illumination to the area of interest in the scene comprises supplying power to only a subset of the flash array, wherein the subset of the flash array is positioned to concentrate the flash illumination on the area of interest in the scene.

3. The method of claim 1 , wherein the device further comprises the flash array.

4. The method of claim 1 , wherein:

the flash array comprises a plurality of light-emitting diodes (LEDs); and

the adjusting comprises providing distinct power levels to respective LEDs of the plurality of LEDs.

5. The method of claim 4 , wherein the providing comprises providing a distinct power level to each LED of the plurality of LEDs.

6. The method of claim 4 , wherein the plurality of LEDs comprises a two- dimensional array of LEDs.

7. The method of claim 4 , wherein the plurality of LEDs comprises two or more LEDs of distinct sizes.

8. The method of claim 1 , wherein redirecting the flash illumination comprises repeatedly varying the power supplied to the respective elements of the flash array to track the predicted movement.

9. The method of claim 1 , wherein the structured light pattern is generated so that one or more of the lines are perpendicular to the edge of the object.

10. A device comprising:

a flash array comprising a plurality of light sources; and

a controller configured to:

construct a depth map of an area of interest in a scene according to one or more images captured while applying a structured light pattern to the area of interest, including:

applying multiple, iterative structured light patterns to the area of interest to obtain multiple respective depth maps, wherein:

the area of interest in the scene comprises an object;

applying the structured light pattern comprises applying one or more lines of the structured light pattern with a specified orientation across an edge of the object so that a subset of the one or more lines of the structured light pattern intersect the edge of the object from at least one side at a specified angle; and

the structured light pattern is rotated for each iteration until the lines of the structured light pattern are determined to be perpendicular to the edge of the object; and

adjust power supplied to respective light sources of the plurality of light sources, to direct flash illumination for a camera to the area of interest in the scene, wherein the controller is configured to predict subsequent movement of the object and redirect flash illumination to the object based on the predicted movement of the object.

11. The device of claim 10 , wherein the flash array is disposed in the camera.

12. The device of claim 10 , wherein:

the plurality of light sources comprises a plurality of LEDs; and

the adjusting comprises providing distinct power levels to respective LEDs of the plurality of LEDs.

13. The device of claim 10 , wherein redirecting the flash illumination comprises repeatedly varying the power supplied to the respective light sources to track the predicted movement.

14. The device of claim 10 , wherein the structured light pattern is generated so that one or more of the lines are perpendicular to the edge of the object.

15. A non-transitory computer-readable storage medium storing one or more programs for execution by one or more processors, the one or more programs comprising instructions for:

constructing a depth map of an area of interest in a scene according to one or more images captured while applying a structured light pattern to the area of interest, including:

applying multiple, iterative structured light patterns to the area of interest to obtain multiple respective depth maps, wherein:

the area of interest in the scene comprises an object;

applying the structured light pattern comprises applying one or more lines of the structured light pattern with a specified orientation across an edge of the object so that a subset of the one or more lines of the structured light pattern intersect the edge of the object from at least one side at a specified angle; and

the structured light pattern is rotated for each iteration until the lines of the structured light pattern are determined to be perpendicular to the edge of the object;

directing flash illumination to the area of interest in the scene, the directing comprising adjusting power supplied to respective elements of a flash array;

capturing an image of the scene as illuminated by the flash illumination;

predicting subsequent movement of the object;

redirecting flash illumination to the object based on the predicted movement of the object; and

capturing one or more additional images as the flash illumination is redirected while the object moves.

16. The non-transitory computer-readable storage medium of claim 15 , wherein redirecting the flash illumination comprises repeatedly varying the power supplied to the respective elements of the flash array to track the predicted movement.

17. The non-transitory computer-readable storage medium of claim 15 , wherein the structured light pattern is generated so that one or more of the lines are perpendicular to the edge of the object.

Assignments (2)
CHANGE OF NAME Recorded Dec 16, 2021
From: FACEBOOK, INC.
To: META PLATFORMS, INC.
Reel/Frame 058520/0535 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 22, 2015
From: SHEN, SHIZHE
To: FACEBOOK, INC.
Reel/Frame 036152/0605 →