Generating images of synthesized bodies wearing a garment
Systems and methods are described for generating images of synthesized bodies wearing a garment. For instance, a source image of a human or mannequin wearing a garment may be submitted to a synthesized human generation system. In response to receiving the source image, the synthesized human generation system may use a classifier to classify the image as depicting one or more body types or orientations. The synthesized human generation system may also apply segmentation to the source image to segment the garment pixels. The synthesized human generation system may then select one or more body generation machine learning models based on the classification of the source image. The synthesized human generation system may utilize the selected machine learning models to generate one or more output images of synthesized bodies that appear to be wearing the garment, using the segmented garment as input.
1 . A system for generating images of a synthesized human body wearing a garment, the system comprising:
a memory configured to store specific computer-executable instructions; and
a processor in communication with the memory and configured to execute the specific computer-executable instructions to at least:
receive a first image, wherein the first image comprises a photograph depicting a human or mannequin wearing the garment;
classify, using an image classifying machine learning model, one or more human body orientations depicted in the first image;
apply segmentation to the first image to generate a second image, the second image comprising pixels depicting the garment from the first image and excluding a plurality of pixels depicting the human or mannequin from the first image;
select, from among a plurality of body generation machine learning models, at least one body generation machine learning model based at least in part on the classified one or more human body orientations depicted in the first image; and
generate, using the at least one body generation machine learning model when provided with the second image as input, one or more generated images depicting a synthesized human wearing the garment, wherein the one or more generated images include the pixels of the second image depicting the garment.
2 . The system of claim 1 , wherein the image classifying machine learning model is trained to identify a body orientation of a mannequin or human in an image.
3 . The system of claim 1 , wherein the one or more human body orientations comprise at least one of a front facing orientation, a back facing orientation, or a side facing orientation.
4 . A computer-implemented method comprising:
receiving a first image, wherein the first image depicts a human or mannequin wearing a garment;
determine a classification of the first image using an image classifier;
applying segmentation to the first image to generate a second image, the second image comprising pixels depicting the garment from the first image;
selecting, from among a plurality of body generation machine learning models, at least one body generation machine learning model based at least in part on the classification of the first image; and
generating, using the at least one body generation machine learning model when provided with the second image as input, one or more generated images depicting a synthesized human wearing the garment.
5 . The computer-implemented method of claim 4 , wherein the one or more generated images depicting a synthesized human wearing the garment are generated based at least in part on user input.
6 . The computer-implemented method of claim 5 , wherein the user input comprises a selection of at least one of a skin tone, hair type, hair color, or facial hair.
7 . The computer-implemented method of claim 5 , wherein the user input comprises a selection of body build to be depicted in the one or more generated images, wherein the body build to be depicted differs from a body build of the human or mannequin depicted in the first image.
8 . The computer-implemented method of claim 4 , further comprising publishing the one or more generated images depicting a synthesized human wearing the garment to an electronic catalog in association with a listing for the garment.
9 . The computer-implemented method of claim 4 , wherein the image classifier is a machine learning model trained to identify a body orientation of a mannequin or human in an image.
10 . The computer-implemented method of claim 4 further comprising training two or more of the plurality of body generation machine learning models, wherein a first model of the plurality of body generation machine learning models is trained using training images depicting clothed humans of a first classification, wherein the second model of the plurality of body generation machine learning models is trained using training images depicting clothed humans of a second classification.
11 . The computer-implemented method of claim 10 , wherein the image classifier is trained to identify at least whether an input image falls within the first classification or the second classification.
12 . The computer-implemented method of claim 10 , wherein the classification determined results in one or more human body orientations depicted in the first image.
13 . The computer-implemented method of claim 12 , wherein the one or more human body orientations comprise at least one of a front facing orientation, a back facing orientation, or a side facing orientation.
14 . A non-transitory computer-readable medium storing specific computer-executable instructions that, when executed by a processor, cause the processor to at least:
receive a first image, wherein the first image depicts a human or mannequin wearing a garment;
determine visual attributes regarding the first image using an image classifier;
apply segmentation to the first image to generate a second image, the second image comprising pixels depicting the garment from the first image;
select, from among a plurality of body generation machine learning models, at least one body generation machine learning model based at least in part on the visual attributes regarding the first image; and
generate, using the at least one body generation machine learning model when provided with the second image as input, one or more generated images depicting a synthesized human wearing the garment.
15 . The non-transitory computer-readable medium of claim 14 , wherein the plurality of body generation machine learning models includes a generative adversarial network (GAN) model.
16 . The non-transitory computer-readable medium of claim 15 , wherein the GAN model comprises (a) a generator that is trained to generate an output image from a photograph of a garment and (b) a discriminator that is trained to determine whether the generated output image appears to be a real image of a clothed human.
17 . The non-transitory computer-readable medium of claim 15 , wherein the GAN model comprises a generator that is trained to generate an image of a clothed human from an input image in which pixels depicting a garment in a photograph have been segmented from other pixels of the photograph.
18 . The non-transitory computer-readable medium of claim 14 , wherein the plurality of body generation machine learning models include a diffusion model.
19 . The non-transitory computer-readable medium of claim 18 , wherein the diffusion model comprises a first encoder and a second encoder, and wherein the first encoder is trained to generate garment features from a garment image and the second encoder is trained to generate latent space of a garment identified in the garment image.
20 . The non-transitory computer-readable medium of claim 19 , wherein the diffusion model is further trained to generate image data depicting a synthesized body within the latent space.