IP Library › Granted Patent US 11,798,202
Granted Patent B2
US 11,798,202 · App. 17/394,064 · Granted Oct 24, 2023

Providing augmented reality-based makeup in a messaging system

Inventors: Riccardo Boscolo (Culver City, CA); Jean Luo (Seattle, WA); Vincent Sung (Los Angeles, CA); Ibram Uppal (Woodland Hills, CA)
Assignee: Snap Inc.
G06T11/00G06Q30/0633G06Q30/0641H04L51/046G06F3/0482G06T2200/24
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,798,202
App. No.
17/394,064
Granted
Oct 24, 2023
Kind
B2
Abstract

Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and method for providing augmented reality-based makeup. The program and method provide for receiving a request to present augmented reality content in association with an image captured by a device camera, the image depicting a user's face; accessing an augmented reality content item applying makeup to the face, the augmented reality content configured to generate a mesh for tracking plural regions of the face and to present available makeup products with respect to the plural regions; presenting the augmented reality content item in association with the face depicted in the image; receiving user input selecting a region of the plural regions; determining a set of available makeup products corresponding to the selected region; and updating presentation of the augmented reality content item based on the set of available makeup products.

Claims (62)

1. A method, comprising:

receiving, by a messaging application running on a device of a user, a request to present augmented reality content in association with an image captured by a device camera, the image depicting a face of the user;

accessing, in response to receiving the request, an augmented reality content item associated with applying makeup to the face, the augmented reality content being configured to generate a mesh for tracking plural regions of the face and to present available makeup products with respect to the plural regions;

presenting the augmented reality content item, including the mesh, in association with the face depicted in the image;

receiving user input selecting a region of the plural regions of the face;

accessing a database storing plural sets of available makeup products corresponding to the selected region, each set of the plural sets of available makeup products having a respective face size and face shape associated therewith;

determining that the respective face size and face shape for a first set of available makeup products of the plural sets of available makeup products matches a size and shape of the face of the user, such that the first set of available makeup products corresponds to the face of the user;

and

updating presentation of the augmented reality content item based on the first set of available makeup products.

2. The method of claim 1 , wherein presenting the augmented reality content item comprises displaying each region of the plural regions as an overlay with respect to the face, the overlay corresponding to an outline of the region.

3. The method of claim 2 , wherein updating presentation of the augmented reality content item comprises displaying an interface for selecting from among the first set of available makeup products.

4. The method of claim 3 , further comprising:

receiving, via the interface, second user input selecting an available makeup product within the first set of available makeup products; and

updating display of the overlay corresponding to the selected region, such that the selected available makeup product is depicted as being applied to the selected region of the face.

5. The method of claim 4 , further comprising:

receiving, via the interface, third user input to add the selected available makeup product to a shopping cart associated with a user account of the user; and

providing, in response to receiving the third user input, for updating the shopping cart with the selected available makeup product.

6. The method of claim 1 , wherein each set of the plural sets of available makeup products is associated with a respective entity.

7. The method of claim 6 , wherein each respective entity is a corporate entity or a social media influencer.

8. The method of claim 1 , further comprising:

displaying, by the messaging application, a carousel interface for selecting the augmented reality content item from among plural augmented reality content items; and

receiving, via the carousel interface, user selection of the augmented reality content item from among the plural augmented reality content items,

wherein the request corresponds to the user selection.

9. A device comprising:

a processor; and

a memory storing instructions that, when executed by the processor, configure the processor to perform operations comprising:

receiving, by a messaging application running on the device, a request to present augmented reality content in association with an image captured by a device camera, the image depicting a face of a user of the device;

accessing, in response to receiving the request, an augmented reality content item associated with applying makeup to the face, the augmented reality content being configured to generate a mesh for tracking plural regions of the face and to present available makeup products with respect to the plural regions;

presenting the augmented reality content item, including the mesh, in association with the face depicted in the image;

receiving user input selecting a region of the plural regions of the face;

accessing a database storing plural sets of available makeup products corresponding to the selected region, each set of the plural sets of available makeup products having a respective face size and face shape associated therewith;

determining that the respective face size and face shape for a first set of available makeup products of the plural sets of available makeup products matches a size and shape of the face of the user, such that the first set of available makeup products corresponds to the face of the user;

and

updating presentation of the augmented reality content item based on the first set of available makeup products.

10. The device of claim 9 , wherein presenting the augmented reality content item comprises display each region of the plural regions as an overlay with respect to the face, the overlay corresponding to an outline of the region.

11. The device of claim 10 , wherein updating presentation of the augmented reality content item comprises displaying an interface for selecting from among the first set of available makeup products.

12. The device of claim 11 , the operations further comprising:

receiving, via the interface, second user input selecting an available makeup product within the first set of available makeup products; and

updating display of the overlay corresponding to the selected region, such that the selected available makeup product is depicted as being applied to the selected region of the face.

13. The device of claim 12 , the operations further comprising:

receiving, via the interface, third user input to add the selected available makeup product to a shopping cart associated with a user account of the user; and

providing, in response to receiving the third user input, for updating the shopping cart with the selected available makeup product.

14. The device of claim 9 , wherein each set of the plural sets of available makeup products is associated with a respective entity.

15. The device of claim 14 , wherein each respective entity is a corporate entity or a social media influencer.

16. The device of claim 9 , the operations further comprising:

displaying, by the messaging application, a carousel interface for selecting the augmented reality content item from among plural augmented reality content items; and

receiving, via the carousel interface, user selection of the augmented reality content item from among the plural augmented reality content items,

wherein the request corresponds to the user selection.

17. A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to perform operations comprising:

receiving, by a messaging application running on a device of a user, a request to present augmented reality content in association with an image captured by a device camera, the image depicting a face of the user;

accessing, in response to receiving the request, an augmented reality content item associated with applying makeup to the face, the augmented reality content being configured to generate a mesh for tracking plural regions of the face and to present available makeup products with respect to the plural regions;

presenting the augmented reality content item, including the mesh, in association with the face depicted in the image;

receiving user input selecting a region of the plural regions of the face;

accessing a database storing plural sets of available makeup products corresponding to the selected region, each set of the plural sets of available makeup products having a respective face size and face shape associated therewith;

determining that the respective face size and face shape for a first set of available makeup products of the plural sets of available makeup products matches a size and shape of the face of the user, such that the first set of available makeup products corresponds to the face of the user;

and

updating presentation of the augmented reality content item based on the first set of available makeup products.

18. The computer-readable storage medium of claim 17 , wherein presenting the augmented reality content item comprises displaying each region of the plural regions as an overlay with respect to the face, the overlay corresponding to an outline of the region.

19. The computer-readable storage medium of claim 18 , wherein updating presentation of the augmented reality content item comprises display an interface for selecting from among the first set of available makeup products.

20. The computer-readable storage medium of claim 19 , the operations further comprising:

receiving, via the interface, second user input selecting an available makeup product within the first set of available makeup products; and

updating display of the overlay corresponding to the selected region, such that the selected available makeup product is depicted as being applied to the selected region of the face.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 23, 2023
From: BOSCOLO, RICCARDO; LUO, JEAN; SUNG, VINCENT; UPPAL, IBRAM
To: SNAP INC.
Reel/Frame 063732/0988 →
Continuity (2)
Provisional Application 63198077 · Sep 28, 2020
Related Publication 20220101566A1 · Mar 31, 2022
Cited By (4)
US 12,361,475 US 12,475,496 US 12,499,593 US 12,548,069