IP Library Granted Patent US 10,013,796
Granted Patent B2
US 10,013,796 · App. 14/603,207 · Granted Jul 3, 2018

Rendering glasses shadows

Inventors: Dmitry Kornilov (San Mateo, CA); Robert Charlie Daugherty (Seattle, WA); Robert Curtis Randolph (Portland, OR); Sergey Surkov (Foster City, CA)
Assignee: Ditto Technologies, Inc.
G06T15/60G06K9/00281G06Q30/0643G06T17/00G06T19/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,013,796
App. No.
14/603,207
Granted
Jul 3, 2018
Kind
B2
Abstract

Rendering glasses with shadows is disclosed, including: generating a face image corresponding to an image of a set of images based at least in part on a face model, wherein the set of images is associated with a user's face; generating a face with shadows image corresponding to the image based at least in part on shadows casted by a glasses model on the face model; generating a shadow transform based at least in part on a difference determined based at least in part on the face image and the face with shadows image; generating a shadowed image based at least in part on applying the shadow transform to the image; and presenting the shadowed image including by overlaying a glasses image associated with the glasses model over the shadowed image.

Claims (64)

1. A system, comprising:

a processor; and

a memory coupled with the processor, wherein the memory is configured to provide the processor with instructions which when executed cause the processor to:

generate a two-dimensional (2D) face image corresponding to an image of a set of images based at least in part on a three-dimensional (3D) face model, wherein the set of images is associated with a user's face;

generate a 2D face with shadows image corresponding to the image based at least in part on shadows casted by a 3D glasses model on the 3D face model, comprising to:

transform the 3D face model and the 3D glasses model using a set of extrinsic information corresponding to the image; and

apply a lighting model to a combination of the transformed 3D face model with the transformed 3D glasses model to cause the transformed 3D glasses model to cast shadows onto the transformed 3D face model, comprising to:

divide an area of the user's face in the image into a predetermined set of segments;

determine a set of lighting parameters associated with the predetermined set of segments; and

determine the lighting model based on the set of lighting parameters;

determine a difference image between the 2D face with shadows image and the 2D face image, comprising to:

subtract the 2D face image from the 2D face with shadows image to obtain the difference image;

generate a shadow transform based at least in part on the difference image, comprising to:

invert dark and light colors of the difference image to obtain the shadow transform;

generate a 2D shadowed image based at least in part on applying the shadow transform to the image, comprising to:

multiply the 2D face image with the shadow transform to obtain the 2D shadowed image; and

present the 2D shadowed image including by overlaying a 2D glasses image associated with the 3D glasses model over the 2D shadowed image.

2. The system of claim 1 , wherein processor is further configured to:

obtain a 3D model associated with the user's face; and

generate a modified 3D generic face model by modifying a 3D generic face model to match the 3D model associated with the user's face, wherein the 3D face model comprises the modified 3D generic face model.

3. The system of claim 2 , wherein the 3D model associated with the user's face is determined from the set of images associated with the user's face.

4. The system of claim 1 , wherein to generate the 2D face image corresponding to the image of the set of images based at least in part on the 3D face model comprises to transform the 3D face model using a set of extrinsic information corresponding to the image.

5. The system of claim 1 , wherein the lighting model is determined based at least in part on the set of images associated with the user's face.

6. The system of claim 1 , wherein to generate the shadow transform further comprises to apply a Despeckle filter to the inverted difference image.

7. The system of claim 1 , wherein to generate the shadow transform further comprises to apply a blur filter to the inverted difference image.

8. The system of claim 1 , wherein to generate the shadow transform further comprises to apply a Red Green Blue (RGB) curves filter to the inverted difference image.

9. The system of claim 1 , wherein to generate the shadow transform further comprises to apply a Hue Saturation Value (HSV) filter adjustment to the inverted difference image.

10. The system of claim 1 , wherein to generate the shadowed image based at least in part on applying the shadow transform to the image comprises multiplying the image with the shadow transform.

11. The system of claim 1 , wherein the 3D glasses model is associated with a pair of glasses selected by a user.

12. A method, comprising:

generating a two-dimensional (2D) face image corresponding to an image of a set of images based at least in part on a three-dimensional (3D) face model, wherein the set of images is associated with a user's face;

generating a 2D face with shadows image corresponding to the image based at least in part on shadows casted by a 3D glasses model on the 3D face model, comprising:

transforming the 3D face model and the 3D glasses model using a set of extrinsic information corresponding to the image; and

applying a lighting model to a combination of the transformed 3D face model with the transformed 3D glasses model to cause the transformed 3D glasses model to cast shadows onto the transformed 3D face model, comprising:

dividing an area of the user's face in the image into a predetermined set of segments;

determining a set of lighting parameters associated with the predetermined set of segments; and

determining the lighting model based on the set of lighting parameters;

determining, using a processor, a difference image between the 2D face with shadows image and the 2D face image, comprising:

subtracting the 2D face image from the 2D face with shadows image to obtain the difference image;

generating a shadow transform based at least in part on the difference image, comprising:

inverting dark and light colors of the difference image to obtain the shadow transform;

generating a 2D shadowed image based at least in part on applying the shadow transform to the image, comprising:

multiplying the 2D face image with the shadow transform to obtain the 2D shadowed image; and

presenting the 2D shadowed image including by overlaying a 2D glasses image associated with the 3D glasses model over the 2D shadowed image.

13. The method of claim 12 , further comprising:

obtaining a 3D model associated with the user's face; and

generating a modified 3D generic face model by modifying a 3D generic face model to match the 3D model associated with the user's face, wherein the 3D face model comprises the modified 3D generic face model.

14. The method of claim 13 , wherein the 3D model associated with the user's face is determined from the set of images associated with the user's face.

15. The method of claim 12 , wherein generating the 2D face image corresponding to the image of the set of images based at least in part on the 3D face model comprises transforming the 3D face model using a set of extrinsic information corresponding to the image.

16. A computer program product, the computer program product being embodied in a non-transitory computer-readable storage medium and comprising computer instructions for:

generating a two-dimensional (2D) face image corresponding to an image of a set of images based at least in part on a three-dimensional (3D) face model, wherein the set of images is associated with a user's face;

generating a 2D face with shadows image corresponding to the image based at least in part on shadows casted by a 3D glasses model on the 3D face model, comprising:

transforming the 3D face model and the 3D glasses model using a set of extrinsic information corresponding to the image; and

applying a lighting model to a combination of the transformed 3D face model with the transformed 3D glasses model to cause the transformed 3D glasses model to cast shadows onto the transformed 3D face model, comprising:

dividing an area of the user's face in the image into a predetermined set of segments;

determining a set of lighting parameters associated with the predetermined set of segments; and

determining the lighting model based on the set of lighting parameters;

determining a difference image between the 2D face with shadows image and the 2D face image, comprising:

subtracting the 2D face image from the 2D face with shadows image to obtain the difference image;

generating a shadow transform based at least in part on the difference image, comprising:

inverting dark and light colors of the difference image to obtain the shadow transform;

generating a 2D shadowed image based at least in part on applying the shadow transform to the image, comprising:

multiplying the 2D face image with the shadow transform to obtain the 2D shadowed image; and

presenting the 2D shadowed image including by overlaying a 2D glasses image associated with the 3D glasses model over the 2D shadowed image.

Assignments (3)
RELEASE OF SECURITY INTEREST Recorded Jun 26, 2023
From: MATERIALISE NV
To: DITTO TECHNOLOGIES, INC.
Reel/Frame 064064/0210 →
SECURITY INTEREST Recorded Aug 14, 2020
From: DITTO TECHNOLOGIES, INC
To: MATERIALISE NV
Reel/Frame 053504/0150 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Apr 1, 2015
From: KORNILOV, DMITRY; DAUGHERTY, ROBERT CHARLIE; RANDOLPH, ROBERT CURTIS; SURKOV, SERGEY
To: DITTO TECHNOLOGIES, INC.
Reel/Frame 035315/0795 →
Continuity (1)
Related Publication 20160217609A1 · Jul 28, 2016