Style profile engine
Various embodiments described herein provide techniques for analyzing an image (e.g., a selfie) of a user with one or more pre-trained machine learned models, to detect facial characteristics of the user. Then, using the output of the machine learned model(s), various combinations of detected facial characteristics are used to assign the user with one style profile of several predefined style profiles. Finally, the style profile may be used in the selection of one or more products to be recommended to the user.
1. A computer-implemented method comprising:
accessing a digital image depicting a face of a user;
inputting the digital image to a first machine learned model;
deriving, by the first machine learned model, output data indicating one or more facial characteristics of one or more facial features of the face depicted in the digital image, the one or more facial characteristics comprising at least a classification of the shape of the face, and a classification of one or more lines of the face;
generating a digital representation of a style profile of the user from the one or more facial characteristics of the one or more facial features of the face;
communicating the digital representation of the style profile of the user to an application executing at a client computing device, the application configured to store the style profile of the user in a memory of the client computing device; and
in response to a user request for a product recommendation, selecting and presenting one or more product recommendations via a user interface of an application executing on the client computing device, wherein the one or more product recommendations are selected based on analysis of the digital representation of the style profile of the user.
2. The computer-implemented method of claim 1 , wherein the output data comprises at least a metric representing a confidence score indicating a likelihood that the shape of the face depicted in the image is one of a predetermined number of shapes that the machine learning model has been trained to detect.
3. The computer-implemented method of claim 2 , wherein the predetermined number of shapes that the machine learning model has been trained to detect comprises one or more of: rectangular, rounded rectangular, circular, ovaloid, inverted triangular, trapezoidal, kite, or any combination thereof.
4. The computer-implemented method of claim 1 , wherein the machine learning model has been trained to output data comprising a metric representing a size of a line detected in the image of the face.
5. The computer-implemented method of claim 1 , wherein the machine learning model has been trained to output data comprising a metric representing a distance between two facial features detected in the image of the face.
6. The computer-implemented method of claim 5 , further comprising:
generating a proportion of a facial feature from the one or more facial characteristics of the one or more facial features of the face.
7. The computer-implemented method of claim 1 , wherein the output data comprises a metric representing a confidence score indicating a likelihood that a line detected in the face depicted in the image is one of a predetermined number of line types that the machine learning model has been trained to detect.
8. The computer-implemented method of claim 7 , wherein the predetermined number of line types that the machine learning model has been trained to detect comprises one or more of: curved and straight.
9. The computer-implemented method of claim 1 , wherein generating a digital representation of a style profile of the user from the one or more facial characteristics of the one or more facial features of the face comprises:
assigning a style profile to the user that maps to a combination of facial characteristics identified in the image of the face of the user.
10. The computer-implemented method of claim 1 , wherein generating a digital representation of a style profile of the user from the one or more facial characteristics of the one or more facial features of the face comprises:
combining a score for each facial characteristic in a combination of facial characteristics associated with a style profile of a plurality of predefined style profiles to derive an aggregate score for the combination of facial characteristics;
comparing the aggregate score for the combination of facial characteristics with a threshold score associated with the style profile; and
assigning the style profile to the user when the aggregate score for the combination of facial characteristics exceeds the threshold score associated with the style profile.
11. The computer-implemented method of claim 1 , further comprising
prompting a user to provide answers to one or more survey questions via a user interface of an application executing at the client computing device; and
generating the digital representation of the style profile of the user based upon answers provided by the user to the one or more survey questions.
12. The computer-implemented method of claim 1 , further comprising:
inputting the digital image to a second machine learned model; and
deriving, by the second machine learned model, color output data relating to one or more colors identified in facial features of the face;
matching a color in the one or more colors identified in facial features of the face to a color palette using a configured color look-up rule; and
communicating the color palette to the mobile application executing at the client computing device;
wherein the one or more product recommendations are selected based on the digital representation of the style of the user and the color palette.
13. A system comprising:
a processor for executing instructions; and
a memory storage device storing instructions, which, when executed by the processor, cause the system to perform operations comprising:
accessing a digital image depicting a face of a user;
inputting the digital image to a first machine learned model;
deriving, by the first machine learned model, output data indicating one or more facial characteristics of one or more facial features of the face depicted in the digital image, the one or more facial characteristics comprising at least a classification of the shape of the face, and a classification of one or more lines of the face;
generating a digital representation of a style profile of the user from the one or more facial characteristics of the one or more facial features of the face;
communicating the digital representation of the style profile of the user to an application executing at a client computing device, the application configured to store the style profile of the user in a memory of the client computing device; and
in response to a user request for a product recommendation, selecting and presenting one or more product recommendations via a user interface of an application executing on the client computing device, wherein the one or more product recommendations are selected based on analysis of the digital representation of the style profile of the user.
14. The system of claim 13 , wherein the output data comprises at least a metric representing a confidence score indicating a likelihood that the shape of the face depicted in the image is one of a predetermined number of shapes that the machine learning model has been trained to detect.
15. The system of claim 14 , wherein the predetermined number of shapes that the machine learning model has been trained to detect comprises one or more of: rectangular, rounded rectangular, circular, ovaloid, inverted triangular, trapezoidal, kite, or any combination thereof.
16. The system of claim 13 , wherein the machine learning model has been trained to output data comprising a metric representing a size of a line detected in the image of the face.
17. The system of claim 13 , wherein the machine learning model has been trained to output data comprising a metric representing a distance between two facial features detected in the image of the face.
18. The system of claim 17 , wherein the operations further comprise:
generating a proportion of a facial feature from the one or more facial characteristics of the one or more facial features of the face.
19. The system of claim 13 , wherein the output data comprises a metric representing a confidence score indicating a likelihood that a line detected in the face depicted in the image is one of a predetermined number of line types that the machine learning model has been trained to detect.
20. The system of claim 19 , wherein the predetermined number of line types that the machine learning model has been trained to detect comprises one or more of: curved and straight.
21. The system of claim 13 , wherein generating a digital representation of a style profile of the user from the one or more facial characteristics of the one or more facial features of the face comprises:
assigning a style profile to the user that maps to a combination of facial characteristics identified in the image of the face of the user.
22. The system of claim 13 , wherein generating a digital representation of a style profile of the user from the one or more facial characteristics of the one or more facial features of the face comprises:
combining a score for each facial characteristic in a combination of facial characteristics associated with a style profile of a plurality of predefined style profiles to derive an aggregate score for the combination of facial characteristics;
comparing the aggregate score for the combination of facial characteristics with a threshold score associated with the style profile; and
assigning the style profile to the user when the aggregate score for the combination of facial characteristics exceeds the threshold score associated with the style profile.
23. The system of claim 13 , further comprising:
prompting a user to provide answers to one or more survey questions via a user interface of an application executing at the client computing device; and
generating the digital representation of the style profile of the user based upon answers provided by the user to the one or more survey questions.
24. The system of claim 13 , further comprising:
inputting the digital image to a second machine learned model; and
deriving, by the second machine learned model, color output data relating to one or more colors identified in facial features of the face;
matching a color in the one or more colors identified in facial features of the face to a color palette using a configured color look-up rule; and
communicating the color palette to the mobile application executing at the client computing device;
wherein the one or more product recommendations are selected based on the digital representation of the style of the user and the color palette.