Garment pattern generation from image data
Systems and methods are provided for generating a flat garment pattern and/or 3D mesh representation of a garment from one or more images depicting the garment laid flat or hung up. A system may obtain both a front image depicting a front view of a garment and a back image depicting a back view of the garment. A front and back silhouette of the garment may then be generated, which may include segmenting the garment depiction from background image content. A parametric representation of the garment may then be generated based on the front and back silhouettes, which may be implemented by iteratively optimizing, using differentiable rendering techniques, a garment representation within a parametric garment space previously learned for the particular garment type. A 3D mesh garment representation may then be generated based on the parametric representation, from which a flat sewing pattern may subsequently be generated if desired.
1. A system comprising:
memory; and
at least one computing device configured with computer-executable instructions that, when executed, cause the at least one computing device to:
obtain (a) a front image depicting a front view of a garment and (b) a back image depicting a back view of the garment, wherein the garment is of a first garment type, wherein the front image and back image each depict the garment laid flat or hung up;
generate a front silhouette and a back silhouette of the garment at least in part by segmenting each of the front image and back image from background image content;
retrieve data defining a parametric garment space previously learned or generated for the first garment type;
generate a parametric representation of the garment based on the front and back silhouettes of the garment, wherein the parametric representation of the garment is generated at least in part by iteratively optimizing a garment representation within the parametric garment space using a differentiable renderer;
generate a three-dimensional (“3D”) mesh representation of the garment based on the parametric representation of the garment; and
generate a flat sewing pattern usable to physically produce one or more instances or variations of the garment, wherein the flat sewing pattern is generated at least in part by applying operations to flatten the 3D mesh representation.
2. The system of claim 1 , wherein the at least one computing device is further configured to:
prior to retrieving the data defining the parametric garment space, learn the parametric garment space, wherein learning the parametric garment space includes at least:
obtaining a sample set of garment patterns of the first garment type;
virtually draping, in virtual 3D space, 3D meshes representing each garment in the sample set of garment patterns to generate a plurality of draped 3D garment meshes;
generating a flattened version of each of the plurality of draped 3D garment meshes;
generating a low-dimensional embedding to represent each flattened version; and
storing and associating with the first garment type, in an electronic data store, data defining the parametric garment space, wherein the parametric garment space contains the low-dimensional embeddings of the sample set.
3. The system of claim 1 , wherein the flat sewing pattern defines two or more flat panels designated to be sewn together at one or more seams to physically produce an instance or variation of the garment.
4. The system of claim 1 , wherein the at least one computing device is further configured to virtually drape the 3D mesh representation of the garment on a 3D representation of a human body to generate a rendered image depicting the garment as worn on the human body.
5. A computer-implemented method comprising:
obtaining (a) a front image depicting a front view of a garment and (b) a back image depicting a back view of the garment, wherein the garment is of a first garment type, wherein the front image and back image each depict the garment laid flat or hung up;
generating a front silhouette of the garment based on the front image and a back silhouette of the garment based on the back image;
generating a parametric representation of the garment based on the front and back silhouettes of the garment, wherein the parametric representation of the garment is generated at least in part by iteratively optimizing a garment representation within a parametric garment space previously learned for the first garment type;
generating a three-dimensional (“3D”) mesh representation of the garment based on the parametric representation of the garment; and
generating a flat pattern for the garment, wherein the flat pattern is generated at least in part by applying operations to flatten the 3D mesh representation.
6. The computer-implemented method of claim 5 , wherein generating the front silhouette and the back silhouette includes segmenting each of the front image and back image from background image content.
7. The computer-implemented method of claim 5 , wherein the parametric representation of the garment is further generated at least in part using differentiable rendering techniques in view of the parametric garment space previously learned for the first garment type.
8. The computer-implemented method of claim 5 , further comprising learning the parametric garment space for the first garment type based in part on a plurality of sample garment patterns for garments of the first garment type.
9. The computer-implemented method of claim 8 , wherein learning the parametric garment space for the first garment type comprises at least:
obtaining a sample set of garment patterns of the first garment type;
virtually draping, in virtual 3D space, 3D meshes representing each garment in the sample set of garment patterns to generate a plurality of draped 3D garment meshes;
generating a flattened version of each of the plurality of draped 3D garment meshes;
generating a low-dimensional embedding to represent each flattened version; and
storing and associating with the first garment type, in an electronic data store, data defining the parametric garment space, wherein the parametric garment space contains the low-dimensional embeddings of the sample set.
10. The computer-implemented method of claim 9 , wherein the low-dimensional embedding representing each flattened version has a lower number of dimensions than its corresponding flattened version.
11. The computer-implemented method of claim 8 further comprising generating at least a subset of the sample set of garment patterns by interpolating one or more reference garment patterns of the first garment type.
12. The computer-implemented method of claim 5 , wherein applying the operations to flatten the 3D mesh representation comprises iteratively compressing the 3D mesh representation along one direction while expanding along at least one orthogonal direction.
13. The computer-implemented method of claim 12 , wherein applying the operations to flatten the 3D mesh representation further comprises, during each of a plurality of iterations of iteratively compressing the 3D mesh representation:
constraining normals of a plurality of triangles of the 3D mesh representation to remain parallel to a compression direction, and
constraining a shape of a two-dimensional (“2D”) view of the 3D mesh representation to correspond to an original shape of the garment as represented in the 3D mesh representation.
14. The computer-implemented method of claim 5 , wherein generating the 3D mesh representation of the garment based on the parametric representation of the garment includes inverting at least one embedding transformation.
15. The computer-implemented method of claim 5 , wherein iteratively optimizing the garment representation within the parametric garment space comprises optimizing over principal component analysis (“PCA”) parameters.
16. The computer-implemented method of claim 5 further comprising virtually draping the 3D mesh representation of the garment on a 3D representation of a human body to generate a rendered image depicting the garment as worn on the human body.
17. Non-transitory computer readable media including computer-executable instructions that, when executed by a computing system, cause the computing system to perform operations comprising:
obtaining (a) a front image depicting a front view of a garment and (b) a back image depicting a back view of the garment, wherein the garment is of a first garment type, wherein the front image and back image each depict the garment laid flat or hung up;
generating a front silhouette of the garment based on the front image and a back silhouette of the garment based on the back image;
generating a parametric representation of the garment based on the front and back silhouettes of the garment, wherein the parametric representation of the garment is generated at least in part by iteratively optimizing, using differentiable rendering techniques, a garment representation within a parametric garment space previously learned for the first garment type; and
generating a three-dimensional (“3D”) mesh representation of the garment based on the parametric representation of the garment.
18. The non-transitory computer readable media of claim 17 , wherein the operations further comprise generating a flat pattern for the garment, wherein the flat pattern is generated at least in part by applying operations to flatten the 3D mesh representation.
19. The non-transitory computer readable media of claim 17 , wherein the operations further comprise virtually draping the 3D mesh representation of the garment on a 3D representation of a human body to generate a rendered image depicting the garment as worn on the human body.
20. The non-transitory computer readable media of claim 17 , wherein the operations further comprise learning the parametric garment space for the first garment type based in part on a plurality of sample garment patterns for garments of the first garment type.