Granted Patent B2

US 12,737,973 · App. 18/527,832 · Granted Sep 15, 2026

Inventors: Omri Berg (Tel Aviv, IL); Itamar Berger (Hod Hasharon, IL); Gal Dudovitch (Tel Aviv, IL); Amir Fruchtman (Holon, IL); Peleg Harel (Ramat Gan, IL)

Assignee: Snap Inc.

G06T17/00G06T15/04

Abstract

Claims (89)

1 . A method comprising:

receiving a plurality of textures associated with an object, each texture of the plurality of textures corresponding to a different view of the object;

automatically generating an initial three-dimensional (3D) model of the object based on an initial alignment of the plurality of textures to respective portions of the initial 3D model;

presenting a view of the initial 3D model in a user interface;

receiving user input via the user interface that adjusts the initial alignment of the plurality of textures to the respective portions of the initial 3D model to provide a revised 3D model, wherein receiving the user input comprises:

receiving a selection of a region of the initial 3D model; and

updating a blending map and a UV map to change an association of the region from a first texture to a second texture of the plurality of textures, the blending map providing continuous blending that gradually changes based on a direction of the view of the initial 3D model;

combining the plurality of textures into a single texture based on the user input that adjusts the initial alignment of the plurality of textures, the single texture defining visual attributes of the object from multiple views; and

storing the revised 3D model in association with the single texture.

2 . The method of claim 1 , wherein the object represents a fashion item, comprising:

computing a pixel value for an individual surface location by multiplying a first pixel value from the first texture by a first weight;

multiplying a second pixel value from the second texture by a second weight; and

summing the multiplied first and second pixel values.

3 . The method of claim 1 , further comprising:

obtaining a particular 3D model of the object;

initializing UV mapping between each of the plurality of textures and corresponding portions of the particular 3D model to provide the initial 3D model;

refining the UV mapping based on the user input; and

generating the single texture based on refining of the UV mapping.

4 . The method of claim 3 , wherein the initial 3D model has a separate UV mapping for each of the plurality of textures and the blending map that associates each surface location of the initial 3D model with one or more of the plurality of textures.

5 . The method of claim 1 , further comprising:

automatically generating the blending map for each surface location of the initial 3D model, the blending map defining an amount of one or more textures that is associated with each surface location.

6 . The method of claim 5 , further comprising:

automatically generating the UV map for each surface location of the initial 3D model, the UV map defining which texture of the plurality of textures is used for the surface location.

7 . The method of claim 6 , further comprising:

associating a first weight with an individual surface location of the initial 3D model, the first weight representing a first amount of the first texture of the plurality of textures;

associating a second weight with the individual surface location of the initial 3D model, the second weight representing a second amount of the second texture of the plurality of textures; and

storing the first and second weights in the blending map in association with the individual surface location.

8 . The method of claim 1 , further comprising:

selecting the first texture of the plurality of textures;

determining an individual view associated with the first texture;

generating a first binary image of the first texture;

generating a second binary image of a view of the object corresponding to the individual view; and

identifying contour key points by matching the first and second binary images.

9 . The method of claim 8 , further comprising:

determining a UV transformation between the first texture and the view of the object based on the contour key points.

10 . The method of claim 1 , wherein receiving the user input comprises:

receiving a selection of the first texture of the plurality of textures; and

in response to the selection of the first texture, enabling modifications to the first texture independently of the second texture of the plurality of textures.

11 . The method of claim 1 , further comprising:

generating a transformation function that deforms the initial 3D model based on the user input.

12 . The method of claim 11 , further comprising:

presenting the view of the initial 3D model in the user interface, the user input being received via the user interface;

determining that the user input corresponds to stretching, shrinking, or changing a position of a texture currently being viewed in the user interface; and

applying the user input to the transformation function to smoothly deform the view of the initial 3D model.

13 . The method of claim 1 , further comprising:

generating a first binary image of the first texture of the plurality of textures;

generating a second binary image of a view of the object; and

identifying contour key points by matching the first and second binary images.

14 . The method of claim 1 , further comprising:

selecting the view of the initial 3D model;

obtaining the direction of a view angle associated with each of the plurality of textures; and

for each surface point of the initial 3D model corresponding to the selected view, computing a pixel value as a function of a dot product between the surface point and the direction of the view angle of each texture of the plurality of textures.

15 . The method of claim 14 , further comprising:

presenting the view of the initial 3D model in the user interface;

receiving, as the user input, a request to change an association between an individual surface point of the initial 3D model from the first texture of the plurality of textures to the second texture of the plurality of textures; and

updating the blending map and UV map based on the request to change the association for the individual surface point.

16 . The method of claim 15 , wherein the request is received using a paintbrush cursor that marks the region of the initial 3D model for which to change the association.

17 . The method of claim 1 , wherein a convolutional neural network (CNN) is used to generate the initial 3D model, the CNN trained by performing training operations comprising:

accessing training data comprising training textures and a training 3D model of a particular object and a ground truth alignment of the training textures to the training 3D model of the particular object;

analyzing the training textures to estimate a mapping of the training textures to the training 3D model;

computing a deviation between the estimated mapping of the training textures to the training 3D model and the ground truth alignment; and

updating one or more parameters of the CNN based on the computed deviation.

18 . A system comprising:

at least one processor; and

at least one memory component having instructions stored thereon that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:

receiving a plurality of textures associated with an object, each texture of the plurality of textures corresponding to a different view of the object;

automatically generating an initial three-dimensional (3D) model of the object based on an initial alignment of the plurality of textures to respective portions of the initial 3D model;

receiving input that adjusts the initial alignment of the plurality of textures to the respective portions of the initial 3D model to provide a revised 3D model;

presenting a view of the initial 3D model in a user interface;

receiving user input via the user interface that adjusts the initial alignment of the plurality of textures to the respective portions of the initial 3D model to provide a revised 3D model, wherein receiving the user input comprises:

receiving a selection of a region of the initial 3D model; and

updating a blending map and a UV map to change an association of the region from a first texture to a second texture of the plurality of textures, the blending map providing continuous blending that gradually changes based on a direction of a view of the initial 3D model;

combining the plurality of textures into a single texture based on the user input that adjusts the initial alignment of the plurality of textures, the single texture defining visual attributes of the object from multiple views; and

storing the revised 3D model in association with the single texture.

19 . The system of claim 18 , wherein a convolutional neural network (CNN) is used to generate the initial 3D model, the CNN trained by performing training operations comprising:

accessing training data comprising training textures and a training 3D model of a particular object and a ground truth alignment of the training textures to the training 3D model of the particular object;

analyzing the training textures to estimate a mapping of the training textures to the training 3D model;

computing a deviation between the estimated mapping of the training textures to the training 3D model and the ground truth alignment; and

updating one or more parameters of the CNN based on the computed deviation.

20 . A non-transitory computer-readable storage medium having stored thereon instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:

receiving a plurality of textures associated with an object, each texture of the plurality of textures corresponding to a different view of the object;

automatically generating an initial three-dimensional (3D) model of the object based on an initial alignment of the plurality of textures to respective portions of the initial 3D model;

receiving input that adjusts the initial alignment of the plurality of textures to the respective portions of the initial 3D model to provide a revised 3D model;

presenting a view of the initial 3D model in a user interface;

receiving user input via the user interface that adjusts the initial alignment of the plurality of textures to the respective portions of the initial 3D model to provide a revised 3D model, wherein receiving the user input comprises:

receiving a selection of a region of the initial 3D model; and

updating a blending map and a UV map to change an association of the region from a first texture to a second texture of the plurality of textures, the blending map providing continuous blending that gradually changes based on a direction of a view of the initial 3D model;

combining the plurality of textures into a single texture based on the user input that adjusts the initial alignment of the plurality of textures, the single texture defining visual attributes of the object from multiple views; and

storing the revised 3D model in association with the single texture.

Assignments (1)

Continuity (1)

References Cited (300)