Systems and methods for using machine learning models to effect virtual try-on and styling on actual users
Disclosed are example embodiments of systems and methods for virtual try-on of articles of clothing. An example method of virtual try-on of articles of clothing includes selecting a garment from a pre-existing database. The method also includes loading a photo of a source model wearing the selected garment. Additionally, the method includes generating a semantic segmentation of the model image. The method also includes extracting the selected garment from the photo of the model. Additionally, the method includes determining a correspondence between a target model and the source model by performing a feature point detection and description of the target model and the source model, and performing feature matching and correspondence validation. The method also includes performing garment warping and alignment of the extracted garment. Additionally, the method includes overlaying and rendering the garment.
1. A method of styling virtual try-on of articles of clothing, the method comprising:
selecting a subset of a large dataset of images of garments from a pre-existing database, wherein the subset of the large data set of images are labelled with a specific style;
generating a digital taxonomy having one or more labels associated with each image in the subset of the large dataset of images, wherein at least one of the one or more labels corresponds to the specific style;
training a style classifier using the digital taxonomy applied to the subset of the large dataset of images of garments; and
generating a first dataset based on executing the trained style classifier on the large dataset of images of garments, the first dataset comprising a plurality of images of garments meeting a threshold probability of depicting the specific style.
2. A method of enhancing an e-commerce retail experience with a shopper avatar, the method comprising:
retrieving a stored outfit selected by a user, the stored outfit comprising at least one garment;
generating a shopper avatar wearing the stored outfit based on a physique of the user;
providing the shopper avatar wearing the stored outfit for display to the user in a user interface;
receiving a selection of a new garment by the user, the selection initiating a virtual try-on of the new garment to replace the at least one garment with the new garment based on the new garment and the at least one garment having a same type of clothing;
configuring the shopper avatar to wear the new garment according to one or more styling options; and
providing the configured shopper avatar wearing the new garment for display to the user in the user interface.
3. The method of claim 2 further comprising:
generating the user interface in a web browser, the user interface overlaid on an online retailer e-commerce website, and
wherein the selection of the new garment by the user comprises a user interaction in the web browser on the online retailer e-commerce website.
4. The method of claim 2 further comprising:
generating the user interface on a mobile device, the user interface overlaid on an online retailer e-commerce application, and
wherein the selection of the new garment by the user comprises a user interaction in the online retailer e-commerce application.
5. The method of claim 2 , wherein each garment represented on the shopper avatar is configurable according to a set of styling options based on a type of clothing.
6. The method of claim 2 , the method further comprising:
receiving a selection of an accessory available on an online retailer e-commerce property;
configuring the shopper avatar to render the accessory based on the physique of the user; and
providing the configured shopper avatar with the accessory for display to the user in the user interface.
7. The method of claim 2 further comprising:
receiving a selection of a second garment available on an online retailer e-commerce property;
configuring the shopper avatar to render the second garment based on the physique of the user; and
providing the configured shopper avatar with the second garment for display to the user in the user interface.
8. The method of claim 7 , wherein the new garment is available on a third-party online retailer different from the online retailer e-commerce property where the second garment is available.
9. A method of enhancing an e-commerce retail experience with on-demand inventory, the method comprising:
receiving a styled configuration of a user-selected garment from a user, the styled configuration based on one or more user interactions with a virtual try-on system;
determining a plurality of recommended fashion items based on the styled configuration of the user-selected garment and available inventory of the plurality of recommended fashion items from at least one online retailer;
assembling a plurality of outfit ensembles based on the plurality of recommended fashion items and the received styled configuration of the user-selected garment to complete a look; and
generating a user interface to cause display of the plurality of outfit ensembles to the user.
10. The method of claim 9 , the method further comprising:
retrieving an image of the styled configuration of the user-selected garment overlaid and rendered on a target model based on a user uploaded photo from a database; and
providing the image of the styled configuration of the user-selected garment overlaid and rendered on the target model based on the user uploaded photo in the user interface.
11. The method of claim 9 , wherein the one or more user interactions with the virtual try-on system comprise:
selecting a styled garment from a catalog of the at least one online retailer;
styling the user-selected garment based on a user selection of one or more styling options associated with the user-selected garment; and
receiving one or more styling recommendations associated with the user-selected garment based on other outfits styled by the user.
12. The method of claim 9 , wherein determining a plurality of recommended fashion items based on the styled configuration of the user-selected garment and available inventory of the plurality of recommended fashion items from at least one online retailer comprises:
retrieving an image of a recommended garment overlaid and rendered on a target model based on a user uploaded photo from a database; and
providing the image of the recommended garment overlaid and rendered on the target model based on the user uploaded photo in the user interface.
13. The method of claim 9 , wherein determining a plurality of recommended fashion items based on the styled configuration of the user-selected garment and available inventory of the plurality of recommended fashion items from at least one online retailer comprises:
selecting a subset of a large dataset of images of available garments from a database, the subset selected based on a clothing type of the user-selected garment, wherein the available garments in the subset are of a different clothing type than the clothing type of the user-selected garment;
ranking the subset of a large dataset of images of available garments from a database based on user-generated metadata; and
determining the plurality of recommended fashion items based on the ranking.
14. The method of claim 13 , the method further comprising:
retrieving availability of the plurality of recommended fashion items from the available inventory of the at least one online retailer based on a physique of the user; and
determining the plurality of recommended fashion items based on the retrieved availability.
15. The method of claim 9 , wherein generating a user interface to cause display of the plurality of outfit ensembles to the user comprises:
for each fashion item in each outfit ensemble:
retrieving a link to the fashion item from a catalog of an online retailer, the link enabling the user to purchase the fashion item from the online retailer;
causing a rendering of the fashion item onto a target model based on a physique of the user;
configuring the rendering of the fashion item based on a set of styling options; and
providing the configured rendering of the fashion item onto the target model and the link to the fashion item in the catalog of the online retailer in the user interface for display to the user on a user device.
16. A method of generating a virtual closet of available digital fashion items, the method comprising:
retrieving a shopper avatar to render in a user interface according to a shopper likeness, the shopper avatar previously configured by a user of a virtual try-on system;
receiving data representing a plurality of garments from a database associated with the virtual try-on system, the data including a stored outfit comprising at least one garment from the plurality of garments;
providing the shopper avatar wearing the stored outfit in the user interface in the virtual try-on system;
generating a user interface element in the user interface, the user interface element comprising a visual representation of a select garment of the plurality of garments;
receiving a user selection of the user interface element; and
rendering the select garment on the shopper avatar for display in the user interface.
17. The method of claim 16 , further comprising:
generating the user interface in a web browser, the user interface overlaid on an online retailer e-commerce website; and
providing a transactional user interface element in the user interface, the transactional user interface element enabling the user to purchase the select garment from the online retailer e-commerce website.
18. The method of claim 16 , further comprising:
generating the user interface on a mobile device, the user interface overlaid on an online retailer e-commerce application; and
providing a transactional user interface element in the user interface, the transactional user interface element enabling the user to purchase the select garment from the online retailer e-commerce application.
19. The method of claim 16 , wherein the plurality of garments includes one or more available garments for purchase from a third-party online retailer, the method further comprising:
retrieving a set of images associated with the one or more available garments from the third-party online retailer;
providing the set of images of the one or more available garments from the third-party online retailer for display in the user interface element in the user interface to the user;
receiving a selection of an image of an available garment from the set of images; and
providing a transactional user interface element in the user interface, the transactional user interface element enabling the user to purchase the available garment from the third-party online retailer.
20. The method of claim 16 , the method further comprising:
receiving a link to a catalog of an online retailer, the link enabling the user to purchase one or more available garments from the online retailer;
retrieving data representing the one or more available garments from the online retailer, the data including a set of images of the one or more available garments, available sizes in stock inventory, and pricing information;
configuring the rendering of each available garment based on the shopper avatar;
providing the set of images associated with the one or more available garments in the user interface element in the user interface;
in response to a selection of a select available garment in the user interface, providing the configured rendering of the select available garment onto the shopper avatar; and
storing the configured rendering of the select available garment onto the shopper avatar and the link to the select available garment in the catalog of the online retailer in the database associated with the virtual try-on system.
21. The method of claim 16 , wherein the data representing the plurality of garments from the database associated with the virtual try-on system comprises historical purchase data from one or more retailers, the method further comprising:
retrieving a set of images associated with one or more previously purchased garments from the one or more retailers;
providing the set of images of the one or more previously purchased garments from the one or more retailers for display in the user interface element in the user interface to the user;
receiving a selection of an image of a previously purchased garment from the set of images; and
providing the image of the previously purchased garment in the user interface element in the user interface.