IP Library › Granted Patent US 11,010,894

Granted Patent B1

US 11,010,894 · App. 16/525,084 · Granted May 18, 2021

Deriving a skin profile from an image

Inventors: Kimberly Shenk (San Rafael, CA); Matthew Camuto (El Cerrito, CA); Donya Fozoonmayeh (San Francisco, CA); Nina Hua (San Francisco, CA); Tatiana Crawford (Denver, CO)

Assignee: NakedPoppy, Inc.

G06T7/0012 A45D44/00 G06K9/6218 G06N20/20 G06T7/90 A45D2044/007 G06T2207/20081 G06T2207/30088 G06T2207/30101

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 11,010,894
App. No.: 16/525,084
Granted: May 18, 2021
Kind: B1

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

Deriving a skin profile from an image is disclosed, including: accessing an input image of a user; deriving a first sub-component based at least in part on the input image of the user; deriving a second sub-component based at least in part on the input image of the user; using a first machine learning model and the first sub-component to determine a predicted base skin color associated with the user; selecting a second machine learning model based at least in part on the predicted base skin color associated with the user; using the second machine learning model and the second sub-component to determine a predicted undertone color associated with the user; determining product recommendations based at least in part on the predicted base skin color associated with the user and the predicted undertone color associated with the user; and presenting the one or more product recommendations at a user interface.

Claims (84)

1. A system, comprising:

a processor configured to:

access an input image of a user, wherein the input image is of a body part that includes a vein;

derive a first sub-component based at least in part on the input image of the user, wherein the first sub-component is associated with the body part without the vein;

derive a second sub-component based at least in part on the input image of the user, wherein the second sub-component is associated with the vein;

derive a third sub-component based at least in part on the input image of the user, wherein the third sub-component is associated with background content without the body part or the vein;

determine a skin dominant color associated with the input image of the user based at least in part on clustering pixel values from the first sub-component;

determine an undertone dominant color associated with the input image of the user based at least in part on clustering pixel values from the second sub-component;

determine a background dominant color associated with the input image of the user based at least in part on clustering pixel values from the third sub-component;

use a first machine learning model and the first sub-component to determine a predicted base skin color associated with the user;

select a second machine learning model based at least in part on the predicted base skin color associated with the user;

use the second machine learning model and the second sub-component to determine a predicted undertone color associated with the user;

obtain user feedback corresponding to the predicted base skin color;

perform machine learning model updating using the user feedback;

determine one or more product recommendations based at least in part on the predicted base skin color associated with the user and the predicted undertone color associated with the user; and

present at least a subset of the one or more product recommendations at a user interface; and

a memory coupled to the processor and configured to provide the processor with instructions.

2. The system of claim 1 , wherein the processor is further configured to:

obtain a lighting adjustment model;

obtain the background dominant color associated with the input image of the user; and

use the lighting adjustment model to determine a lighting adjustment associated with the input image of the user based at least in part on the background dominant color associated with the input image.

3. The system of claim 1 , wherein the first machine learning model comprises a plurality of cluster centers corresponding to respective ones of a plurality of skin colors, and wherein to use the first machine learning model and the first sub-component to determine the predicted base skin color associated with the user comprises to:

input the skin dominant color into the first machine learning model to determine a matching cluster center; and

determine a skin color associated with the matching cluster center as the predicted base skin color.

4. The system of claim 1 , wherein the second machine learning model comprises a plurality of cluster centers corresponding to respective ones of a plurality of undertone colors corresponding to the predicted base skin color associated with the user, and wherein to use the second machine learning model and the second sub-component to determine the predicted undertone color associated with the user comprises to:

input the undertone dominant color into the second machine learning model to determine a matching cluster center; and

determine an undertone color associated with the matching cluster center as the predicted undertone color.

5. The system of claim 1 , wherein to perform the machine learning model updating using the user feedback comprises to update the first machine learning model based at least in part on the user feedback corresponding to the predicted base skin color.

6. The system of claim 1 , wherein the processor is further configured to obtain user feedback corresponding to the predicted undertone color.

7. The system of claim 6 , wherein the processor is further configured to update the second machine learning model based at least in part on the user feedback corresponding to the predicted undertone color.

8. A method, comprising:

accessing an input image of a user, wherein the input image is of a body part that includes a vein;

deriving a first sub-component based at least in part on the input image of the user, wherein the first sub-component is associated with the body part without the vein;

deriving a second sub-component based at least in part on the input image of the user, wherein the second sub-component is associated with the vein;

deriving a third sub-component based at least in part on the input image of the user, wherein the third sub-component is associated with background content without the body part or the vein;

determining a skin dominant color associated with the input image of the user based at least in part on clustering pixel values from the first sub-component;

determining an undertone dominant color associated with the input image of the user based at least in part on clustering pixel values from the second sub-component;

determining a background dominant color associated with the input image of the user based at least in part on clustering pixel values from the third sub-component;

using a first machine learning model and the first sub-component to determine a predicted base skin color associated with the user;

selecting a second machine learning model based at least in part on the predicted base skin color associated with the user;

using the second machine learning model and the second sub-component to determine a predicted undertone color associated with the user;

obtaining user feedback corresponding to the predicted base skin color;

performing machine learning model updating using the user feedback;

determining one or more product recommendations based at least in part on the predicted base skin color associated with the user and the predicted undertone color associated with the user; and

presenting at least a subset of the one or more product recommendations at a user interface.

9. The method of claim 8 , further comprising:

obtaining a lighting adjustment model;

obtaining the background dominant color associated with the input image of the user; and

using the lighting adjustment model to determine a lighting adjustment associated with the input image of the user based at least in part on the background dominant color associated with the input image.

10. The method of claim 8 , wherein the first machine learning model comprises a plurality of cluster centers corresponding to respective ones of a plurality of skin colors, and wherein using the first machine learning model and the first sub-component to determine the predicted base skin color associated with the user comprises:

inputting the skin dominant color into the first machine learning model to determine a matching cluster center; and

determining a skin color associated with the matching cluster center as the predicted base skin color.

11. The method of claim 8 , wherein the second machine learning model comprises a plurality of cluster centers corresponding to respective ones of a plurality of undertone colors corresponding to the predicted base skin color associated with the user, and wherein using the second machine learning model and the second sub-component to determine the predicted undertone color associated with the user comprises:

inputting the undertone dominant color into the second machine learning model to determine a matching cluster center; and

determining an undertone color associated with the matching cluster center as the predicted undertone color.

12. The method of claim 8 , wherein performing the machine learning model updating using the user feedback comprises updating the first machine learning model based at least in part on the user feedback corresponding to the predicted base skin color.

13. The method of claim 8 , further comprising obtaining user feedback corresponding to the predicted undertone color.

14. A computer program product, the computer program product being embodied in a non-transitory computer-readable storage medium and comprising computer instructions for: