IP Library › Granted Patent US 12700198

Granted Patent B1

US 12700198 · App. 18/755,499 · Granted Aug 4, 2026

Makeup blending in image generation

Inventors: Hui Pang (Bothell, WA); Sunil Sharadchandra Hadap (Dublin, CA); Rahul Suresh (Vancouver, CA); Amin Banitalebi Dehkordi (Vancouver, CA); Violetta Shevchenko (Adelaide, AU)

Assignee: Amazon Technologies, Inc.

G06T19/006 G06T5/50 G06V10/60 G06T2207/10024 G06T2207/20221 G06T2207/30201

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 12700198
App. No.: 18/755,499
Granted: Aug 4, 2026
Kind: B1

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

Systems and methods are provided for generating an image of a person's face virtually wearing a selected makeup given an input image of the face. The face image can be processed to decompose aspects of the person's face, such as shading, highlighting, boundaries, and color of the face. In order to estimate the resulting color of the face wearing makeup, the systems and methods may estimate reflectance of the skin in the face image and of the selected makeup. The resulting color of the face wearing the makeup may be estimated using reflectance of both the skin and makeup color. Then, the resulting image can be reconstructed from the decomposed aspects of the person's face by incorporating the shading and highlighting of the face with the estimated resulting color within the boundaries of the face. The result is a realistic image of the person virtually wearing the selected makeup.

Claims (58)

1 . A computer-implemented method comprising:

receiving a face image depicting a face upon which to virtually apply makeup;

processing the face image to generate face image preprocessing results;

obtaining a makeup transmittance for the makeup;

determining a makeup reflectance for the makeup indicating how light is reflected off of the makeup;

determining a skin reflectance for the face indicating how light is reflected off of skin on the face;

determining, based on the makeup transmittance, the makeup reflectance, and the skin reflectance, a layering reflectance representing reflectance of the makeup layered on top of the skin on the face;

generating, using the layering reflectance, a blended albedo image depicting the skin blended with the makeup on the face; and

generating, using the blended albedo image and the face image preprocessing results, a result image depicting the face in the face image wearing the makeup.

2 . The computer-implemented method of claim 1 , wherein the face image preprocessing results comprise a face semantic segmentation mask, an albedo layer, a highlight layer, and a shadow layer.

3 . The computer-implemented method of claim 1 , wherein obtaining the makeup transmittance comprises estimating a wavelength curve of the transmittance based on a thickness of the makeup and using a scattering coefficient.

4 . The computer-implemented method of claim 1 , wherein the makeup reflectance is estimated using a scattering coefficient and a thickness of the makeup.

5 . The computer-implemented method of claim 1 , wherein the layering reflectance is a wavelength function representing a spectrum of light.

6 . The computer-implemented method of claim 1 , wherein the layering reflectance is determined using a modified Kubelka-Munk theory.

7 . The computer-implemented method of claim 6 , wherein the modified Kubelka-Munk theory comprises applying an expansion of an integral representing the layering reflectance.

8 . The computer-implemented method of claim 1 , further comprising:

displaying, in association with a user device that includes a user device camera, the result image, wherein the face image comprises a face image of the user captured in real time by the user device camera.

9 . The computer-implemented method of claim 1 , further comprising:

causing display, on a user device via a computer application, the result image.

10 . The computer-implemented method of claim 1 , further comprising:

obtaining a second makeup transmittance for a second makeup;

determining a second makeup reflectance for the second makeup indicating how light is reflected off of the second makeup;

determining, based on the makeup transmittance, the makeup reflectance, the second makeup transmittance, the second makeup reflectance, and the skin reflectance, a second layering reflectance representing reflectance of the makeup and the second makeup layered on top of the skin on the face;

generating, using the second layering reflectance, a second blended albedo image depicting the skin blended with the makeup and the second makeup on the face; and

generating, using the second blended albedo image and the face image preprocessing results, a second result image depicting the face in the face image wearing the makeup and the second makeup.

11 . A non-transitory computer-readable medium storing specific computer-executable instructions that, when executed by a processor, cause the processor to at least:

receive a face image depicting a face upon which to virtually apply makeup;

process the face image to generate face image preprocessing results;

determine a makeup transmittance for the makeup;

determine a makeup reflectance for the makeup indicating how light is reflected off of the makeup;

determine a skin reflectance for the face indicating how light is reflected off of skin on the face;

determine, based on the makeup transmittance, the makeup reflectance, and the skin reflectance, a layering reflectance representing reflectance of the makeup layered on top of the skin on the face;

generate, using the layering reflectance, a blended albedo image depicting the skin blended with the makeup on the face; and

generate, using the blended albedo image and the face image preprocessing results, a result image depicting the face in the face image wearing the makeup.

12 . The non-transitory computer-readable medium of claim 11 , wherein the face image preprocessing results comprise a face semantic segmentation mask, an albedo layer, a highlight layer, and a shadow layer.

13 . The non-transitory computer-readable medium of claim 11 , wherein the makeup transmittance and the makeup reflectance are estimated using a thickness of the makeup and a scattering coefficient.

14 . The non-transitory computer-readable medium of claim 11 , wherein the layering reflectance is determined using a modified Kubelka-Munk theory.

15 . The non-transitory computer-readable medium of claim 14 , wherein the modified Kubelka-Munk theory comprises applying an expansion of an integral representing the layering reflectance.

16 . The non-transitory computer-readable medium of claim 11 , wherein the computer-executable instructions, when executed by the processor, further cause the processor to:

obtain a second makeup transmittance for a second makeup;

determine a second makeup reflectance for the second makeup indicating how light is reflected off of the second makeup;

determine, based on the makeup transmittance, the makeup reflectance, the second makeup transmittance, the second makeup reflectance, and the skin reflectance, a second layering reflectance representing reflectance of the makeup and the second makeup layered on top of the skin on the face;

generate, using the second layering reflectance, a second blended albedo image depicting the skin blended with the makeup and the second makeup on the face; and

generate, using the second blended albedo image and the face image preprocessing results, a second result image depicting the face in the face image wearing the makeup and the second makeup.

17 . A system comprising:

one or more processors; and

a memory that stores computer-executable instructions, wherein the computer-executable instructions, when executed, cause the one or more processors to: