Using machine learning models to generate a mirror representing an image of virtual try-on and styling of an actual user
Disclosed are example implementations of systems and methods for virtual try-on of articles of clothing. An example method of virtual try-on of articles of clothing includes ingesting information specifying articles of clothing using a first machine learning model; retrieving a stored outfit, the stored outfit comprising at least one garment; inferring a style for the stored outfit preferred by the user with a second machine learning model; generating a shopper avatar wearing the stored outfit with the inferred style based on a physique of a user; and providing the shopper avatar wearing the stored outfit with the inferred style for display to the user in a user interface.
1. A method comprising:
ingesting, using a server, information specifying articles of clothing using a first machine learning model to generate a database of garments, each garment in the database having a plurality of labels, at least one label being one or more styling combinations;
receiving, from a user, selection of a first body type;
determining a first garment based on user interaction with a user interface of a platform for purchasing clothing having a virtual try-on capability;
determining, using a server, an inferred style based upon the first garment with a second machine learning model, wherein the inferred style includes at least one styling combination;
retrieving, from the database of garments, one or more additional garments, a footwear, and an accessory based upon the inferred style and the first garment and combining the one or more additional garments, the footwear, the accessory and the first garment to create a stored outfit with a look, wherein the look is garments, footwear, and an accessory stylized to a selected preference of the user in the at least one styling combination;
generating a shopper avatar having a second body type matching the first body type and wearing the stored outfit with the inferred style showing the look, wherein the first garment and the one or more additional garments are warped and aligned using a third machine learning model to deform the first garment and the one or more additional garments over the shopper avatar including showing a clothing interaction of the first garment with the one or more additional garments; and
providing the shopper avatar wearing the stored outfit with the inferred style showing the look including the first garment and the one or more additional garments are warped and aligned and the clothing interaction for display in a user interface in real-time so that the user interface serves as a mirror through which the user can see what they would look like in real life.
2. The method of claim 1 , wherein the styling combination includes one or more from a group of a fashion standard styling segment, a top styling segment having one of from a group of a following styling segments including sleeves buttoned, half tucked, fully buttoned front portion and collar folded, sleeve length styling segment having potential length values of wrist, mid hand and fingers, and bodice fit styling segment having potential fit values of tight, relaxed and oversized.
3. The method of claim 1 , further comprising:
receiving a selection of a new garment by the user, the selection initiating a virtual try-on of the new garment to replace the first garment with the new garment based on the new garment and the first garment having a same type of clothing;
configuring the shopper avatar to wear the new garment according to one or more styling options; and
providing the shopper avatar wearing the new garment for display to the user in the user interface.
4. The method of claim 1 , further comprising:
selecting an additional garment of clothing to recommend to the user using the second machine learning model in accordance with the inferred style of the user;
updating the user interface to include the additional garment in the inferred style along with the shopper avatar; and
providing the updating the user interface for display to the user.
5. The method of claim 1 , wherein the information specifying articles of clothing is from one or more from a group of: a designer, a retailer, a B2B customer, a website, a B2B application, and a mobile application.
6. The method of claim 1 , wherein ingesting information specifying articles of clothing using the first machine learning model is performed using artificial intelligence and machine learning algorithms and a rendering engine.
7. The method of claim 1 , wherein ingesting information specifying articles of clothing using the first machine learning model, further comprises:
selecting a subset of a large dataset of images of garments from a pre-existing database, wherein the subset of the large data set of images are labelled with a specific style;
generating a digital taxonomy having one or more labels associated with each image in the subset of the large dataset of images, wherein at least one of the one or more labels corresponds to the specific style;
training a style classifier using the digital taxonomy applied to the subset of the large dataset of images of garments; and
generating a first dataset based on executing the trained style classifier on the large dataset of images of garments, the first dataset comprising a plurality of images of garments meeting a threshold probability of depicting the specific style.
8. The method of claim 1 , wherein the shopper avatar is a rendering of the user or a human model.
9. The method of claim 1 , wherein receiving, from a user, selection of a first body type further comprises:
presenting a user interface with a plurality of human body types;
receiving a selection of one of the plurality of human body types; and
presenting the selection of the one of the plurality of human body types as the first body type for the shopper avatar in the user interface along with body dimensions.
10. The method of claim 9 , wherein receiving the selection of one of the plurality of human body types comprises:
receiving a photograph or a video of the user;
analyzing the photograph or video of the user; and
selecting one of the plurality of human body types as the first body type based on the analyzing the photograph or video of the user.
11. The method of claim 1 , wherein the second machine learning model infers the inferred style for the stored outfit preferred by the user based upon styling guidance and recommendations for the user based on a current and historical profile and current and historical engagements of the user.
12. The method of claim 1 , wherein the user interface in a window overlaid on a web page of a web browser showing in the web page of a website of a retailer.
13. The method of claim 1 , wherein the warping and aligning the one garment over the shopper avatar uses a warping model and a synthesis model.
14. A system comprising one or more processors and memory operably coupled with the one or more processors, wherein the memory stores instructions that, in response to execution of the instructions by the one or more processors, cause the one or more processors to perform the following operations:
ingesting information specifying articles of clothing using a first machine learning model to generate a database of garments, each garment in the database having a plurality of labels, at least one label being one or more styling combinations;
receiving, from a user, selection of a first body type;
determining a first garment based on user interaction with a user interface of a platform for purchasing clothing having a virtual try-on capability;
determining an inferred style based upon the first garment with a second machine learning model, wherein the inferred style includes at least one styling combination;
retrieving, from the database of garments, one or more additional garments, a footwear, and an accessory based upon the inferred style and the first garment and combining the one or more additional garments, the footwear, the accessory and the first garment to create a stored outfit with a look, wherein the look is garments, footwear, and an accessory stylized to a selected preference of the user in the at least one styling combination;
generating a shopper avatar having a second body type matching the first body type and wearing the stored outfit with the inferred style showing the look, wherein the first garment and the one or more additional garments are warped and aligned using a third machine learning model to deform the first garment and the one or more additional garments over the shopper avatar including showing a clothing interaction of the first garment with the one or more additional garments; and
providing the shopper avatar wearing the stored outfit with the inferred style showing the look including the first garment and the one or more additional garments are warped and aligned and the clothing interaction for display in a user interface in real-time so that the user interface serves as a mirror through which the user can see what they would look like in real life.
15. The system of claim 14 , wherein the styling combination includes one or more from a group of a fashion standard styling segment, a top styling segment having one of from a group of a following styling segments including sleeves buttoned, half tucked, fully buttoned front portion and collar folded, sleeve length styling segment having potential length values of wrist, mid hand and fingers, and bodice fit styling segment having potential fit values of tight, relaxed and oversized.
16. The system of claim 14 , wherein the operations further comprise:
receiving a selection of a new garment by the user, the selection initiating a virtual try-on of the new garment to replace the first garment with the new garment based on the new garment and the first garment having a same type of clothing;
configuring the shopper avatar to wear the new garment according to one or more styling options; and
providing the shopper avatar wearing the new garment for display to the user in the user interface.
17. The system of claim 14 , wherein the operations further comprise:
selecting an additional garment of clothing to recommend to the user using the second machine learning model in accordance with the inferred style of the user;
updating the user interface to include the additional garment in the inferred style along with the shopper avatar; and
providing the updating the user interface for display to the user.
18. The system of claim 14 , wherein the information specifying articles of clothing is from one or more from a group of: a designer, a retailer, a B2B customer, a website, a B2B application, and a mobile application.
19. The system of claim 14 , wherein ingesting information specifying articles of clothing using the first machine learning model is performed using artificial intelligence and machine learning algorithms and a rendering engine.
20. The system of claim 14 , wherein ingesting information specifying articles of clothing using the first machine learning model further comprises the operations of:
selecting a subset of a large dataset of images of garments from a pre-existing database, wherein the subset of the large data set of images are labelled with a specific style;
generating a digital taxonomy having one or more labels associated with each image in the subset of the large dataset of images, wherein at least one of the one or more labels corresponds to the specific style;
training a style classifier using the digital taxonomy applied to the subset of the large dataset of images of garments; and
generating a first dataset based on executing the trained style classifier on the large dataset of images of garments, the first dataset comprising a plurality of images of garments meeting a threshold probability of depicting the specific style.
21. The system of claim 14 , wherein the shopper avatar is a rendering of the user or a human model.
22. The system of claim 14 , wherein receiving, from a user, selection of a first body type further comprises:
presenting a user interface with a plurality of human body types;
receiving a selection of one of the plurality of human body types; and
presenting the selection of the one of the plurality of human body types as the first body type for the shopper avatar in the user interface along with body dimensions.
23. The system of claim 22 , wherein receiving the selection of one of the plurality of human body types comprises:
receiving a photograph or a video of the user;
analyzing the photograph or video of the user; and
selecting one of the plurality of human body types as the first body type based on the analyzing the photograph or video of the user.
24. The system of claim 14 , wherein the second machine learning model infers the style for the stored outfit preferred by the user based upon styling guidance and recommendations for the user based on a current and historical profile and current and historical engagements of the user.
25. The system of claim 14 , wherein the user interface in a window overlaid on a web page of a web browser showing in the web page of a website of a retailer.
26. The system of claim 14 , wherein the warping and aligning the one garment over the shopper avatar uses a warping model and a synthesis model.