IP Library › Granted Patent US 10,339,365

Granted Patent B2

US 10,339,365 · App. 15/086,749 · Granted Jul 2, 2019

Automated avatar generation

Inventors: Maksim Gusarov (Venice, CA); Igor Kudriashov (Venice, CA); Valerii Filev (Venice, CA); Sergei Kotcur (Venice, CA)

Assignee: Snap Inc.

G06K9/00281 G06T13/40

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 10,339,365
App. No.: 15/086,749
Granted: Jul 2, 2019
Kind: B2

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

Systems, devices, media, and methods are presented for generating facial representations using image segmentation with a client device. The systems and methods receive an image depicting a face, detect at least a portion of the face within the image, and identify a set of facial landmarks within the portion of the face. The systems and methods determine one or more characteristics representing the portion of the face, in response to detecting the portion of the face. Based on the one or more characteristics and the set of facial landmarks, the systems and methods generate a representation of a face.

Claims (98)

1. A method, comprising:

receiving, by one or more processors, one or more two-dimensional images depicting at least a portion of a face of a first user;

detecting, by the one or more processors, the portion of the face depicted within the one or more two-dimensional images;

identifying a set of facial landmarks within the portion of the face depicted within the one or more two-dimensional images;

in response to detecting the portion of the face and the set of facial landmarks, determining one or more characteristics representing the portion of the face depicted in the one or more two-dimensional images, the determining comprising:

identifying a set of colors within a hair region of the face;

determining a hair texture for the hair region based on variations in the set of colors that indicate edges, objects or shapes within the hair associated with the hair texture;

comparing a dimension of the hair region to one or more of the set of facial landmarks; and

determining hair length or hair volume based on the comparison of the dimension of the hair region to the one or more of the set of facial landmarks;

based on the one or more characteristics and the set of facial landmarks, generating a representation of a face for the at least one portion of the face depicted in the one or more images;

generating one or more sticker graphics incorporating the representation of the face, the generating of the one or more sticker graphics comprising generating a scaled version of the representation corresponding to a template graphic; and

generating, by a user device of the first user, a message, directed to a second user, that includes the one or more sticker graphics incorporating the representation of the face of the first user.

2. The method of claim 1 , wherein identifying the set of facial landmarks further comprises:

determining one or more distances between two or more facial landmarks of the set of facial landmarks.

3. The method of claim 2 , wherein determining the distances further comprises:

determining a first distance between facial landmarks of the set of facial landmarks representing eyes depicted on the portion of the face;

determining a second distance between facial landmarks of the set of facial landmarks representing the eyes and a nose depicted on the portion of the face;

determining a third distance between facial landmarks of the set of facial landmarks representing the eyes and a mouth depicted on the portion of the face; and

determining a fourth distance between facial landmarks of the set of facial landmarks representing the eyes and a chin depicted on the portion of the face.

4. The method of claim 2 , wherein determining the one or more characteristics further comprises:

determining a gender of the portion of the face based on the one or more distances between the two or more facial landmarks.

5. The method of claim 2 , wherein determining the one or more characteristics further comprises:

determining a race identifier of the portion of the face based on the one or more distances between the two or more facial landmarks; and

determining a skin color by identifying an area of interest on the portion of the face and extracting an average color depicted within the area of interest.

6. The method of claim 2 , wherein determining the one or more characteristics further comprises:

determining a jaw shape of the portion of the face based on the set of facial landmarks and the one or more distances between the two or more facial landmarks; and

fitting a polyline to the jaw shape.

7. The method of claim 2 , wherein determining the one or more characteristics further comprises:

identifying one or more iris within the portion of the face; and

determining a prevailing color for the one or more iris.

8. The method of claim 7 , wherein identifying the one or more iris further comprises:

determining a shape of one or more eyes surrounding the one or more iris;

determining a height of the shape based on the set of facial landmarks; and

determining an iris dimension based on the height of the shape of the one or more eyes.

9. The method of claim 2 , wherein determining the one or more characteristics further comprises:

determining one or more eyebrow region of the portion of the face; and

identifying one or more shape of the one or more eyebrow region.

10. The method of claim 2 , wherein determining the one or more characteristics further comprises:

determining a lip region of the portion of the face;

identifying a shape of one or more lips within the lip region; and

identifying a prevailing color of the lip region.

11. The method of claim 1 , wherein:

the hair volume is determined by:

identifying a first portion of the set of facial landmarks that represents an outline of the face; and

computing a distance the hair region extends from the identified portion of the set of facial landmarks that represents the outline of the face to an outer opposing edge of the hair region; and

the hair length is determined based on the hair volume and by:

identifying a second portion of the set of facial landmarks that represents a chin; and

computing a distance from the hair region to the second portion of the set of facial landmarks that represents the chin.

12. The method of claim 1 , wherein determining the one or more characteristics comprises:

selecting a region of the portion of the face where obstructions commonly appear:

determining whether an obstruction is present in the selected region using one or more of the set of facial landmarks; and

in response to determining that the obstruction is present, matching the obstruction to a template selected from a set of templates.

13. A system, comprising:

one or more processors; and

a non-transitory processor-readable storage medium coupled to the one or more processors and storing processor executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:

receiving, by the one or more processors, one or more two-dimensional images depicting at least a portion of a face of a first user;