Granted Patent B2

US 12,363,253 · App. 17/323,895 · Granted Jul 15, 2025

Inventors: Alexander J. Wilson (Seattle, WA); Tom Neckermann (Seattle, WA); Romain Gabriel Paul Rey (Vancouver, CA)

Assignee: Microsoft Technology Licensing, LLC

H04N5/272G06F18/211G06N3/08G06N20/00G06T7/73G06T2207/20084H04N2005/2726

Abstract

Claims (80)

1. A system comprising:

one or more processors; and

computer storage memory having computer-executable instructions stored thereon which, when executed by the one or more processors, implement a method comprising:

obtaining a first set of images of a first user, a first image of the first set of images including a first representation of a first personal style of the first user and a second image of the first set of images including a second representation of a second personal style of the first user;

causing a clustering model to classify a first subset of images of the first set of images based on the first personal style;

encoding a feature vector representing the first personal style based on the first subset of images;

causing a machine learning model to be trained to generate synthetic images including representations of the first user with the first personal style based on the feature vector, the machine learning model trained by:

causing the machine learning model to generate a set of synthetic images including representations of the first user with the first personal style based on a second subset of images of the first set of images depicting the first user without the first personal style;

causing a first discriminator to generate a first set of predictions indicating whether the synthetic images of the set of synthetic images depict the first personal style;

causing a second discriminator to generate a second set of predictions indicating whether the synthetic images of the set of synthetic images are generated by the machine learning model; and

modifying a set of parameters of the machine learning model based on the first set of predictions and the second set of predictions;

receiving a second set of images of the first user;

detecting, using a third image of the second set of images, a feature associated with the first user;

causing a user interface element representing the first personal style to be displayed in a user interface; and

in response to a user interaction with the user interface element, causing the machine learning model to generate the synthetic images representing the first personal style of the first user by at least modifying a subset of images of the second set of images based on the first personal style.

2. The system of claim 1 , wherein the first personal style of the first user corresponds to data indicating a style of a group of styles consisting of: a hair style of the first user, a makeup style of the first user, a facial hair style of the first user, and a clothing style of the first user.

3. The system of claim 1 , wherein the first set of images includes a snapshot digital photograph of the first user.

4. The system of claim 1 , wherein the first set of images includes a video feed of the first user.

5. The system of claim 1 , wherein the second set of images includes a video feed of the first user.

6. The system of claim 1 , the method further comprising:

receiving a third set of images of the first user, the third set of images are indicative of the first user without the first personal style;

determining, by comparing the first subset of images of the first set of images of the first user to a second subset of images of the third set of images, a difference between the first subset of images of the first set of images and the second subset of images of the third set of images; and

based on the determining of the difference, determining the first personal style.

7. The system of claim 1 , the method further comprising:

obtaining a plurality of images of the first user, at least a third image of the plurality of images, has a same personal style of the first user relative to a second image of the plurality of images, at least a fourth image of the plurality of images, has a different personal style of the first user relative to the third image of the plurality of images and the second image of the plurality of images;

clustering the third image of the plurality of images and the second image of the plurality of images into the first personal style;

clustering the fourth image of the plurality of images into the second personal style; and

based on the clustering, cause presentation, to a user device, of an indicator that is user-selectable to allow the first user to apply the first personal style or the second personal style to the second set of images.

8. The system of claim 1 , the method further comprising anonymizing, based on one or more predefined rules, one or more portions of the first set of images such that only a portion of the first set of images are received.

9. The system of claim 1 , the method further comprising:

causing presentation of a first user interface element;

in response to receiving an indication the first user interface element has been selected, adjusting an intensity of data indicative of rendering a particular quantity of the first personal style of the first user; and

in response to the adjusting, feeding intensity adjustments into the machine learning model to fine-tune the machine learning model.

10. The system of claim 1 , the method further comprising:

receiving a third set of images of a second user;

in response to the receiving, detecting, based on at least one image of the third set of images, a second feature vector representing the second user; and

based on the first personal style of the first user, causing the machine learning model to apply the first personal style of the first user to the second feature vector.

11. The system of claim 1 , the method further comprising:

determining second data indicative of the second personal style of the first user;

in response to the receiving of the second set of images, detecting, using the second set of images, one or more second features of the first user;

generating a first user interface element; and

based on the determining of the second data and in response to receiving an indication that the first user interface element has been selected, applying the second data indicative of the second personal style of the first user to the one or more second features of the first user in the second set of images, wherein the second set of images includes the first personal style.

12. A computer-implemented method comprising:

obtaining a first set of images of a first user, a first image of the first set of images including a first representation of a first personal style of the first user and a second image of the first set of images including a second representation of a second personal style of the first user;

causing a clustering model to classify the first image of the first set of images based on the first personal style and the second image of the first set of images based on the second personal style;

determining first data indicative of the first personal style of the first user, wherein the first data includes a first feature vector representing the first personal style based on the first image;

determining second data indicative of the second personal style of the first user, wherein the second data includes a second feature vector representing the second personal style based on the second image;

causing a machine learning model to be trained by:

causing the machine learning model to generate a set of synthetic images including representations of the first user with the first personal style based on a first subset of images of the first set of images depicting the first user without the first personal style;

causing a first discriminator to generate a first set of predictions indicating whether synthetic images of the set of synthetic images depict the first personal style;

causing a second discriminator to generate a second set of predictions indicating whether the synthetic images of the set of synthetic images are generated by the machine learning model; and

modifying a set of parameters of the machine learning model based on the first set of predictions and the second set of predictions;

generating a first user interface element associated with the first personal style of the first user;

generating a second user interface element associated with the second personal style of the first user;

receiving a second set of images of the first user;

receiving an indication that the first user interface element has been selected; and

in response to the receiving of the indication, causing the machine learning model to apply the first data indicative of the first personal style of the first user to a third feature vector of the first user generated based on the second set of images.

13. The computer-implemented method of claim 12 , wherein the first set of images includes a snapshot digital photograph of the first user.

14. The computer-implemented method of claim 12 , wherein the second set of images includes a video feed of the first user.

15. The computer-implemented method of claim 12 , further comprising:

receiving a third set of images of the first user, the third set of images depicting the first user without the first personal style;

determining, by comparing the first subset of images of the first set of images of the first user to a second subset of images of the third set of images, a difference between the first subset of images of the first set of images and the second subset of images of the third set of images; and

based on the determining of the difference, determining the first personal style.

16. The computer-implemented method of claim 12 , further comprising anonymizing, based on one or more predefined rules, one or more portions of the first set of images such that only a portion of the set of first images are received.

17. The computer-implemented method of claim 12 , further comprising:

generating a third user interface element;

in response to receiving a second indication the third user interface element has been selected, adjusting an intensity of data indicative of rendering a particular quantity of the first personal style of the first user.

18. One or more computer storage media having computer-executable instructions embodied thereon that, when executed, by one or more processors, cause the one or more processors to perform a method, the method comprising:

obtaining a first set of images of a first user, a first image of the first set of images including a first representation of a first personal style of the first user and a second image of the first set of images including a second representation of a second personal style of the first user;

causing a clustering model to classify the first image of the first set of images based on the first personal style;

determining data indicative of the first personal style of the first user, wherein the data indicative of the first personal style of the first user includes a feature vector generated based on the first image;

receiving a second set of images of at least one of: the first user and a second user;

in response to the receiving, generating, using a second image of the second set of images, a second feature vector of at least one of: the first user and the second user; and

based on the determining, applying a set of data that resembles the first personal style of the first user to the second feature vector, wherein the set of data is generated by a machine learning model trained by:

causing the machine learning model to generate a set of synthetic images including representations of the first user with the first personal style based on a subset of images of the first set of images depicting the first user without the first personal style;

causing a first discriminator to generate a first set of predictions indicating whether synthetic images of the set of synthetic images depict the first personal style;

causing a second discriminator to generate a second set of predictions indicating whether the synthetic images of the set of synthetic images are generated by the machine learning model; and

modifying a set of parameters of the machine learning model based on the first set of predictions and the second set of predictions.

19. The one or more computer storage media of claim 18 , wherein the data indicative of the first personal style of the first user corresponds to data indicating a style of a group of styles consisting of: a hair style of the first user, a makeup style of the first user, a facial hair style of the first user, and a clothing style of the first user.

20. The one or more computer storage media of claim 18 , where the first set of images includes a video feed of the first user.

Assignments (1)

Continuity (1)

References Cited (17)