IP Library › Granted Patent US 12688627
Granted Patent B2
US 12688627 · App. 17/751,393 · Granted Jul 21, 2026

Using augmented face images to improve facial recognition tasks

Inventors: Shuo Cheng (Los Angeles, CA); Guoxian Song (Los Angeles, CA); Wanchun Ma (Los Angeles, CA); Chao Wang (Los Angeles, CA); Linjie Luo (Los Angeles, CA)
Assignee: Lemon Inc.
G06V40/172G06T11/60G06V10/761G06V10/82
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12688627
App. No.
17/751,393
Filed
May 23, 2022
Granted
Jul 21, 2026
Kind
B2
Art Unit
2665
USPC
382/181
Abstract

Methods and systems for generating synthetic images based on an input image are described. The method may include receiving an input image; generating, using an encoder, a first latent code vector representation based on the input image; receiving a latent code corresponding to a feature to be added to the input image; modifying the first latent code vector representation based on the latent code corresponding to the feature to be added; generating, by an image decoder, a synthesized image based on the modified first latent code vector representation; identifying, using a landmark detector, one or more landmarks in the base image; identifying, using a landmark detector, one or more landmarks in the synthesized image; determining a measure of similarity between the landmark identified on the base image and the landmark identified in the synthesized image; and discarding the synthesized image based on the comparison.

Claims (59)

1 . A method for generating synthetic images based on an input image, the method comprising:

receiving an input image;

generating, using an encoder, a first latent code vector representation based on the input image;

determining a feature underrepresented in a base image set by analyzing the base image set;

receiving a latent code corresponding to the feature to be added to the input image;

modifying the first latent code vector representation based on the latent code corresponding to the feature to be added to the input image;

generating, by an image decoder, a synthesized image based on the modified first latent code vector representation;

identifying, using a landmark detector, one or more landmarks in the input image;

identifying, using a landmark detector, one or more landmarks in the synthesized image;

determining a measure of similarity between the landmark identified on the input image and the landmark identified in the synthesized image; and

discarding the synthesized image based on the measure of similarity,

wherein the determining a feature underrepresented in a base image set further comprises:

analyzing a plurality of input images to determine at least one underrepresented feature of a plurality of underrepresented features; and

identifying the feature to be added to the input image based on the analysis.

2 . The method of claim 1 , wherein the latent code corresponding to the feature to be added is a randomly-scaled semantic vector.

3 . The method of claim 1 , wherein the feature to be added corresponds to at least one of a beard, eyeglasses, earrings, or a mustache.

4 . The method of claim 1 , wherein the image decoder is a StyleGAN image decoder.

5 . The method of claim 1 , further comprising:

training a machine learning neural network model utilizing a plurality of synthesized images, wherein the generated synthesized image is included in the plurality of synthesized images.

6 . The method of claim 1 , wherein the input image includes a facial image of a person.

7 . A system, comprising:

one or more hardware processors configured by machine-readable instructions to:

receive an input image;

generate, using an encoder, a first latent code vector representation based on the input image;

determine a feature underrepresented in a base image set by analyzing the base image set;

receive a latent code corresponding to the feature to be added to the input image;

modify the first latent code vector representation based on the latent code corresponding to the feature to be added to the input image;

generate, by an image decoder, a synthesized image based on the modified first latent code vector representation;

identify, using a landmark detector, one or more landmarks in the input image;

identify, using a landmark detector, one or more landmarks in the synthesized image;

determine a measure of similarity between the landmark identified on the input image and the landmark identified in the synthesized image; and

discard the synthesized image based on the measure of similarity,

wherein the determine a feature underrepresented in a base image set further comprises:

analyzing a plurality of input images to determine at least one underrepresented feature of a plurality of underrepresented features; and

identifying the feature to be added to the input image based on the analysis.

8 . The system of claim 7 , wherein the latent code corresponding to the feature to be added is a randomly-scaled semantic vector.

9 . The system of claim 7 , wherein the feature to be added corresponds to at least one of a beard, eyeglasses, earrings, or a mustache.

10 . The system of claim 7 , further comprising: training a machine learning neural network model utilizing a plurality of synthesized images, wherein the generated synthesized image is included in the plurality of synthesized images.

11 . The system of claim 7 , wherein the input image includes a facial image of a person.

12 . The system of claim 7 , wherein the image decoder is a StyleGAN image decoder.

13 . A non-transitory computer-readable storage medium comprising instructions being executable by one or more processors to cause the one or more processors to:

receive an input image;

generate, using an encoder, a first latent code vector representation based on the input image;

determine a feature underrepresented in a base image set by analyzing the base image set;

receive a latent code corresponding to the feature to be added to the input image;

modify the first latent code vector representation based on the latent code corresponding to the feature to be added to the input image;

generate, by an image decoder, a synthesized image based on the modified first latent code vector representation;

identify, using a landmark detector, one or more landmarks in the input image;

identify, using a landmark detector, one or more landmarks in the synthesized image;

determine a measure of similarity between the landmark identified on the input image and the landmark identified in the synthesized image; and

discard the synthesized image based on the measure of similarity,

wherein the determine a feature underrepresented in a base image set further comprises:

analyzing a plurality of input images to determine at least one underrepresented feature of a plurality of underrepresented features; and

identifying the feature to be added to the input image based on the analysis.

14 . The computer-readable storage medium of claim 13 , wherein the latent code corresponding to the feature to be added is a randomly-scaled semantic vector.

15 . The computer-readable storage medium of claim 13 , wherein the determine a feature underrepresented in a base image set further comprises: training a machine learning neural network model utilizing a plurality of synthesized images, wherein the generated synthesized image is included in the plurality of synthesized images.

16 . The computer-readable storage medium of claim 13 , wherein the input image includes a facial image of a person.

17 . The computer-readable storage medium of claim 13 , wherein the image decoder is a StyleGAN image decoder.

18 . The computer-readable storage medium of claim 13 , wherein the feature to be added corresponds to at least one of a beard, eyeglasses, earrings, or a mustache.