IP Library Granted Patent US 11,455,829
Granted Patent B2
US 11,455,829 · App. 16/796,497 · Granted Sep 27, 2022

System, method, and computer program for capturing an image with correct skin tone exposure

Inventors: William Guie Rivard (Menlo Park, CA); Brian J. Kindle (Sunnyvale, CA); Adam Barry Feder (Mountain View, CA)
Assignee: DUELIGHT LLC
G06V40/168G06T5/008G06T7/40G06T7/50G06T17/20G06V10/22G06V10/50G06V10/758G06T2207/10024G06T2207/20012G06T2207/20208G06T2207/30201
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,455,829
App. No.
16/796,497
Granted
Sep 27, 2022
Kind
B2
Abstract

A system and method are provided for capturing an image with correct skin tone exposure. In use, one or more faces are detected having threshold skin tone within a scene. Next, based on the detected one or more faces, the scene is segmented into one or more face regions and one or more non-face regions. A model of the one or more faces is constructed based on a depth map and a texture map, the depth map including spatial data of the one or more faces, and the texture map includes surface characteristics of the one or more faces. The one or more images of the scene are captured based on the model. Further, in response to the capture, the one or more face regions are processed to generate a final image.

Claims (40)

1. A device, comprising:

a non-transitory memory storing instructions;

a camera module; and

one or more processors in communication with the non-transitory memory and the camera module, wherein the one or more processors execute the instructions to:

cause the camera module to capture one or more images of a scene;

detect one or more faces having threshold skin tone within the scene;

based on the detected one or more faces, segment the scene into one or more face regions and one or more non-face regions;

construct a model of the one or more faces based on a depth map and a texture map, the depth map including at least spatial data of the one or more faces, and the texture map includes surface characteristics of the one or more faces;

modify a contrast condition associated with visible skin for each of the detected one or more faces based on the model; and

process the one or more face regions to generate a final image.

2. The device of claim 1 , wherein the threshold skin tone is a predefined average intensity or histogram median.

3. The device of claim 2 , wherein the predefined average intensity or histogram median is based on the one or more face regions.

4. The device of claim 1 , wherein the device is operable such that the one or more images include at least one high dynamic range (HDR) image.

5. The device of claim 1 , wherein the device is operable such that the one or more images include two or more images such that a first image of the two or more images is exposed so that a median intensity of the one or more face regions defines a mid-point intensity of the first image, and a second image of the two or images is exposed so that a median intensity of the one or more non-face regions defines a mid-point intensity of the second image.

6. The device of claim 5 , wherein the device is operable such that the processing includes processing the one or more non-face regions and applying an HDR effect by blending, using a strength coefficient, the first image and the second image.

7. The device of claim 1 , wherein the device is operable such that the capturing is of a video stream.

8. The device of claim 1 , wherein the device is operable such that the processing includes applying a local equalization on pixels within the one or more face regions.

9. The device of claim 1 , wherein the device is operable such that a second model is constructed of one or more individuals depicted within the scene based on the depth map and at least one visual image.

10. The device of claim 9 , wherein the depth map and the texture map are used to construct a three-dimensional (3D) model of the one or more faces.

11. The device of claim 9 , wherein the at least one visual image includes a surface texture.

12. The device of claim 11 , wherein the surface texture includes one or more features associated with each of the one or more faces.

13. The device of claim 1 , wherein the device is operable such that the segmenting includes a transition region between the one or more face regions and the one or more non-face regions.

14. The device of claim 1 , wherein the surface characteristics include at least one of color, moles, cuts, scars, freckles, or facial hair.

15. The device of claim 1 , wherein the device is operable such that a lighting gradient is used to modify post-processing lighting conditions on the one or more faces.

16. The device of claim 1 , wherein the device is operable to further execute the instructions to correct a color of light associated with a tone of the visible skin associated with each of the one or more faces based on a measurement of an ambient lighting color balance.

17. The device of claim 1 , wherein the device is operable such that a histogram associated with the final image shows correct exposure data for the scene.

18. A computer-implemented method, comprising:

capturing, using a camera, one or more images of a scene;

detecting, using a processor, one or more faces having threshold skin tone within the scene;

based on the detected one or more faces, segmenting, using the processing, the scene into one or more face regions and one or more non-face regions;

constructing a model of the one or more faces based on a depth map and a texture map, the depth map including at least spatial data of the one or more faces, and the texture map includes surface characteristics of the one or more faces;

modifying a contrast condition associated with visible skin for each of the detected one or more faces based on the model; and

processing, using the processor, the one or more face regions to generate a final image.

19. A computer program product comprising computer executable instructions stored on a non-transitory computer readable medium that when executed by a processor instruct the processor to:

cause a camera module to capture one or more images of a scene;

detect one or more faces having threshold skin tone within the scene;

based on the detected one or more faces, segment the scene into one or more face regions and one or more non-face regions;

construct a model of the one or more faces based on a depth map and a texture map, the depth map including at least spatial data of the one or more faces, and the texture map includes surface characteristics of the one or more faces;

modify a contrast condition associated with visible skin for each of the detected one or more faces based on the model; and

process the one or more face regions to generate a final image.

Assignments (2)
SECURITY INTEREST Recorded Jul 30, 2021
From: DUELIGHT, LLC
To: ZILKA KOTAB, P.C.
Reel/Frame 057043/0301 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 24, 2020
From: RIVARD, WILLIAM GUIE; KINDLE, BRIAN J.; FEDER, ADAM BARRY
To: DUELIGHT LLC
Reel/Frame 051901/0835 →
Cited By (6)
US 12,211,153 US 12,401,911 US 12,401,912 US 12,418,727 US 12,445,736 US 12,666,159