High quality AR cosmetics simulation via image filtering techniques
The disclosure is directed to embodiments for producing high-quality images simulating the application of a material (e.g., virtual cosmetics) to a person's body. Example implementations can generate an augmented image displaying a virtual cosmetic layer (e.g., lipstick) on a person's face. For instance, a computing system can obtain an image depicting the face and identify a region for applying the cosmetic. The system can use augmented reality and/or image filtering to process the image into datasets that can be combined with material data related to the virtual cosmetic to generate augmented image(s) simulating application of the material.
1. A computing system comprising:
one or more processors;
one or more non-transitory computer readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
obtaining an image of a face;
obtaining material property data associated with a material, wherein the material property data comprises data associated with a mid-tones spectral band;
adjusting a saturation of at least a portion of the image to generate a resulting image;
generating a mid-tones image dataset based at least in part on the resulting image and a base color value, wherein the base color value comprises a color pixel value for at least a portion of the image, wherein the mid-tones image dataset is associated with the mid-tones spectral band;
processing the mid-tones image dataset with the material property data to generate a mid-tones map; and
generating, based at least in part on the mid-tones map, an augmented image that depicts the material applied to the face.
2. The system of claim 1 , wherein generating a mid-tones image dataset comprises:
multiplying the resulting image by the base color value.
3. The system of claim 2 , wherein the base color value comprises a red-green-blue pixel value.
4. The system of claim 1 , wherein generating a mid-tones image dataset comprises:
adjusting one or more image levels based at least in part on a gamma value associated with the material property data.
5. The system of claim 4 , wherein the one or more image levels comprise one or more brightness levels associated with a histogram of brightness from black to white.
6. The system of claim 1 , wherein the operations further comprise:
generating a highlights image dataset, wherein generating the highlights image dataset comprises:
adjusting one or more image levels of the resulting image based on an input black and a gamma value associated with the material property data.
7. The system of claim 1 , wherein the operations further comprise:
adjusting one or more tritones of the resulting image based at least in part on an input gray color associated with the material property data;
adjusting one or more image levels of the resulting image based at least in part on an input gamma value associated with the material property data; and
generating a shadows image dataset based on the resulting image on a weighted base color value associated with a particular region of the image.
8. A computer-implemented method, the method comprising:
obtaining, by a computing system comprising one or more processors, an image, wherein the image comprises a face;
obtaining, by the computing system, material property data associated with a material for application on one or more facial features;
decomposing, by the computing system, the image into a plurality of image datasets respectively associated with a plurality of spectral bands, wherein the plurality of image datasets comprise: a shadows image dataset associated with a shadows spectral band, a highlights image dataset associated with a highlights spectral band, and a mid-tones image dataset associated with a mid-tones spectral band;
processing, by the computing system, the plurality of image datasets and the material property data to generate a plurality of material maps, wherein the plurality of material maps comprise one or more material maps for each respective spectral band of the plurality of spectral bands; and
blending, by the computing system, the plurality of material maps with the image to generate an augmented image.
9. The method of claim 8 , wherein processing the plurality of image datasets and the material property data to generate the plurality of material maps comprises:
processing, by the computing system, the shadows image dataset and material property data associated with the shadows spectral band to generate a first material map;
processing, by the computing system, the highlights image dataset and material property data associated with the highlights spectral band to generate a second material map; and
processing, by the computing system, the mid-tones image dataset and material property data associated with the mid-tones spectral band to generate a third material map.
10. The method of claim 8 , further comprising:
detecting, by the computing system, a location of the one or more facial features of the face.
11. The method of claim 8 , further comprising:
providing, by the computing system, the augmented image via an interface of an augmented reality application.
12. The method of claim 8 , wherein the augmented image is descriptive of a simulation of the material applied to the face.
13. The method of claim 8 , wherein the material is associated with a cosmetic product.
14. The method of claim 8 , wherein the material property data is descriptive of at least one of a color, reflectivity, or opacity associated with the material.
15. The method of claim 8 , wherein the plurality of material maps are associated with the one or more facial features.
16. One or more non-transitory computer-readable media that store instructions that, when executed by one or more processors, cause the one or more processors to perform operations, the operations comprising:
obtaining image data, wherein the image data comprises an object;
obtaining material property data associated with a material for application on the object;
decomposing the image data into a plurality of image datasets respectively associated with a plurality of spectral bands, wherein the plurality of image datasets comprise: a shadows image dataset associated with a shadows spectral band, a highlights image dataset associated with a highlights spectral band, and a mid-tones image dataset associated with a mid-tones spectral band;
processing the plurality of image datasets and the material property data to generate a plurality of material maps, wherein the plurality of material maps comprise one or more material maps for each respective spectral band of the plurality of spectral bands; and
generating an augmented image data based on the plurality of material maps and the image data.
17. The one or more non-transitory computer-readable media of claim 16 , wherein the image data comprises a video from a camera.
18. The one or more non-transitory computer-readable media of claim 16 , further comprising:
providing the augmented image data via a real-time simulation interface provided by an online retailer.
19. The one or more non-transitory computer-readable media of claim 16 , wherein the object comprises a body part.
20. The one or more non-transitory computer-readable media of claim 16 , wherein the operations further comprise:
performing a face tracking technique on the image data to generate face tracking data; and
wherein the augmented image data is generated based at least in part on the face tracking data.