IP Library › Granted Patent US 12,039,675
Granted Patent B2
US 12,039,675 · App. 17/587,148 · Granted Jul 16, 2024

High quality AR cosmetics simulation via image filtering techniques

Inventors: Kanstantsin Sokal (San Francisco, CA); Siarhei Kazakou (Mountain View, CA); Igor Kibalchich (Mountain View, CA); Matsvei Zhdanovich (Zurich, CH)
Assignee: GOOGLE LLC
G06T19/006G06T3/18G06T5/00G06T7/11G06T7/246G06T7/73G06T15/503G06V40/161G06T2207/20081G06T2207/30201
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,039,675
App. No.
17/587,148
Granted
Jul 16, 2024
Kind
B2
Abstract

The disclosure is directed to embodiments for producing high-quality images simulating the application of a material (e.g., virtual cosmetics) to a person's body. Example implementations can generate an augmented image displaying a virtual cosmetic layer (e.g., lipstick) on a person's face. For instance, a computing system can obtain an image depicting the face and identify a region for applying the cosmetic. The system can use augmented reality and/or image filtering to process the image into datasets that can be combined with material data related to the virtual cosmetic to generate augmented image(s) simulating application of the material.

Claims (53)

1. A computing system comprising:

one or more processors;

one or more non-transitory computer readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:

obtaining an image of a face;

obtaining material property data associated with a material, wherein the material property data comprises data associated with a mid-tones spectral band;

adjusting a saturation of at least a portion of the image to generate a resulting image;

generating a mid-tones image dataset based at least in part on the resulting image and a base color value, wherein the base color value comprises a color pixel value for at least a portion of the image, wherein the mid-tones image dataset is associated with the mid-tones spectral band;

processing the mid-tones image dataset with the material property data to generate a mid-tones map; and

generating, based at least in part on the mid-tones map, an augmented image that depicts the material applied to the face.

2. The system of claim 1 , wherein generating a mid-tones image dataset comprises:

multiplying the resulting image by the base color value.

3. The system of claim 2 , wherein the base color value comprises a red-green-blue pixel value.

4. The system of claim 1 , wherein generating a mid-tones image dataset comprises:

adjusting one or more image levels based at least in part on a gamma value associated with the material property data.

5. The system of claim 4 , wherein the one or more image levels comprise one or more brightness levels associated with a histogram of brightness from black to white.

6. The system of claim 1 , wherein the operations further comprise:

generating a highlights image dataset, wherein generating the highlights image dataset comprises:

adjusting one or more image levels of the resulting image based on an input black and a gamma value associated with the material property data.

7. The system of claim 1 , wherein the operations further comprise:

adjusting one or more tritones of the resulting image based at least in part on an input gray color associated with the material property data;

adjusting one or more image levels of the resulting image based at least in part on an input gamma value associated with the material property data; and

generating a shadows image dataset based on the resulting image on a weighted base color value associated with a particular region of the image.

8. A computer-implemented method, the method comprising:

obtaining, by a computing system comprising one or more processors, an image, wherein the image comprises a face;

obtaining, by the computing system, material property data associated with a material for application on one or more facial features;

decomposing, by the computing system, the image into a plurality of image datasets respectively associated with a plurality of spectral bands, wherein the plurality of image datasets comprise: a shadows image dataset associated with a shadows spectral band, a highlights image dataset associated with a highlights spectral band, and a mid-tones image dataset associated with a mid-tones spectral band;

processing, by the computing system, the plurality of image datasets and the material property data to generate a plurality of material maps, wherein the plurality of material maps comprise one or more material maps for each respective spectral band of the plurality of spectral bands; and

blending, by the computing system, the plurality of material maps with the image to generate an augmented image.

9. The method of claim 8 , wherein processing the plurality of image datasets and the material property data to generate the plurality of material maps comprises:

processing, by the computing system, the shadows image dataset and material property data associated with the shadows spectral band to generate a first material map;

processing, by the computing system, the highlights image dataset and material property data associated with the highlights spectral band to generate a second material map; and

processing, by the computing system, the mid-tones image dataset and material property data associated with the mid-tones spectral band to generate a third material map.

10. The method of claim 8 , further comprising:

detecting, by the computing system, a location of the one or more facial features of the face.

11. The method of claim 8 , further comprising:

providing, by the computing system, the augmented image via an interface of an augmented reality application.

12. The method of claim 8 , wherein the augmented image is descriptive of a simulation of the material applied to the face.

13. The method of claim 8 , wherein the material is associated with a cosmetic product.

14. The method of claim 8 , wherein the material property data is descriptive of at least one of a color, reflectivity, or opacity associated with the material.

15. The method of claim 8 , wherein the plurality of material maps are associated with the one or more facial features.

16. One or more non-transitory computer-readable media that store instructions that, when executed by one or more processors, cause the one or more processors to perform operations, the operations comprising:

obtaining image data, wherein the image data comprises an object;

obtaining material property data associated with a material for application on the object;

decomposing the image data into a plurality of image datasets respectively associated with a plurality of spectral bands, wherein the plurality of image datasets comprise: a shadows image dataset associated with a shadows spectral band, a highlights image dataset associated with a highlights spectral band, and a mid-tones image dataset associated with a mid-tones spectral band;

processing the plurality of image datasets and the material property data to generate a plurality of material maps, wherein the plurality of material maps comprise one or more material maps for each respective spectral band of the plurality of spectral bands; and

generating an augmented image data based on the plurality of material maps and the image data.

17. The one or more non-transitory computer-readable media of claim 16 , wherein the image data comprises a video from a camera.

18. The one or more non-transitory computer-readable media of claim 16 , further comprising:

providing the augmented image data via a real-time simulation interface provided by an online retailer.

19. The one or more non-transitory computer-readable media of claim 16 , wherein the object comprises a body part.

20. The one or more non-transitory computer-readable media of claim 16 , wherein the operations further comprise:

performing a face tracking technique on the image data to generate face tracking data; and

wherein the augmented image data is generated based at least in part on the face tracking data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 24, 2022
From: SOKAL, KANSTANTSIN; KAZAKOU, SIARHEI; KIBALCHICH, IGOR; ZHDANOVICH, MATSVEI
To: GOOGLE LLC
Reel/Frame 059391/0063 →
Continuity (2)
Continuation 16692267 · Nov 22, 2019
Related Publication 20220157030A1 · May 19, 2022