Automatic rig creation process
The disclosure provides methods and systems for automatically generating an animatable object, such as a 3D model. In particular, the present technology provides fast, easy, and automatic animatable solutions based on unique facial characteristics of user input. Various embodiments of the present technology include receiving user input, such as a two-dimensional image or three-dimensional scan of a user's face, and automatically detecting one or more features. The methods and systems may further include deforming a template geometry and a template control structure based on the one or more detected features to automatically generate a custom geometry and custom control structure, respectively. A texture of the received user input may also be transferred to the custom geometry. The animatable object therefore includes the custom geometry, the transferred texture, and the custom control structure, which follow a morphology of the face.
1. A method for automatically generating an animatable object, the method comprising:
receiving user input for a face, the user input comprising at least one of an image, a video signal, and a 3D scan indicative of the face;
automatically detecting one or more features of the received user input for the face, the automatically detecting comprising automatically determining a plurality of spatial coordinates for the face, each spatial coordinate of the plurality of spatial coordinates for the face being associated with one of the one or more features of the received user input for the face, the plurality of spatial coordinates for the face being determined using ray casting techniques;
automatically deforming a template geometry based on the one or more detected features to automatically generate a custom geometry;
automatically transferring a texture of the received user input to the custom geometry;
automatically deforming a template control structure based on the one or more detected features to automatically generate a custom control structure; and
automatically generating an animatable object having the custom geometry, the transferred texture, and the custom control structure.
2. The method as recited in claim 1 , wherein a set of features of the template geometry corresponds to the one or more detected features, the deforming the template geometry including matching the one or more detected features to the set of features of the template geometry.
3. The method as recited in claim 2 , wherein the deforming the template geometry to automatically generate the custom geometry uses radial basis functions.
4. The method as recited in claim 1 , wherein a set of control elements of the template control structure corresponds to a subset of the one or more detected features, the deforming the template control structure including matching the subset of the one or more detected features to the set of control elements of the template control structure.
5. The method as recited in claim 1 , further comprising receiving a user-defined geometry, wherein the template geometry is the user-defined geometry.
6. The method as recited in claim 1 , further comprising receiving a user-defined control structure, wherein the template control structure is the user-defined control structure.
7. A system for automatically generating an animatable object, the system comprising:
a processor; and
a memory for storing executable instructions, the processor executing the instructions to:
receive user input for a face including at least one of an image, a video signal, and a 3D scan indicative of the face;
automatically detect one or more features of the received user input for the face, the automatically detect comprising automatically determining a plurality of spatial coordinates for the face, each spatial coordinate of the plurality of spatial coordinates for the face being associated with one of the one or more features of the received user input for the face, the plurality of spatial coordinates for the face being determined using ray casting techniques comprising each detected feature being projected from an artificial 2D plane onto a template 3D model via a ray passing from an origin through a respective 2D coordinate of the detected feature;
automatically deform a template geometry based on the one or more detected features to automatically generate a custom geometry;
automatically transfer a texture of the received user input to the custom geometry;
automatically deform a template control structure based on the one or more detected features to automatically generate a custom control structure; and
automatically generate an animatable object having the custom geometry, the transferred texture, and the custom control structure.
8. The system as recited in claim 7 , wherein the user input is received from a client device via a network.
9. The system as recited in claim 7 , wherein the processor executes the instructions to receive a user-defined geometry from a client device and to store the user-defined geometry associated with the client device, wherein the template geometry is the user-defined geometry.
10. The system as recited in claim 7 , wherein the processor executes the instructions to receive a user-defined control structure from a client device and to store the user-defined control structure associated with the client device, wherein the template control structure is the user-defined control structure.
11. The system as recited in claim 7 , wherein vertices of the custom geometry are based on one or more of the plurality of spatial coordinates determined from the one or more detected features of the received user input.
12. The system as recited in claim 7 , wherein control elements of the custom control structure are based on spatial coordinates determined from the one or more detected features of the received user input.
13. A method for automatically generating an animatable 3D model, the method comprising:
receiving user input for a face including at least one of an image, a video signal, and a 3D scan indicative of the face;
automatically detecting one or more features of the received user input for the face, the automatically detecting comprising automatically determining a plurality of spatial coordinates for the face, each spatial coordinate of the plurality of spatial coordinates for the face being associated with one of the one or more features of the received user input for the face, the plurality of spatial coordinates for the face being determined using ray casting techniques comprising each detected feature being projected from an artificial 2D plane onto a template 3D model via a ray passing from an origin through a respective 2D coordinate of the detected feature;
automatically determining one or more first spatial coordinates, each first spatial coordinate associated with one of the one or more detected features;
automatically deforming a template geometry based on the one or more first spatial coordinates to automatically generate a custom geometry;
automatically transferring a texture of the received user input to the custom geometry;
automatically deforming a template control structure based on a subset of the one or more first spatial coordinates to automatically generate a custom control structure; and
automatically generating an animatable 3D model having the custom geometry, the transferred texture, and the custom control structure.
14. The method as recited in claim 13 , wherein the user input is indicative of at least one of a face and a body of a user.
15. The method as recited in claim 13 , wherein the user input is a video signal, the automatically detecting one or more features, the deforming the template geometry, the transferring the texture, and the deforming the template control structure being performed in real time.
16. The method as recited in claim 13 , wherein the transferring the texture to the custom geometry includes automatically mapping at least one pixel of the texture to a corresponding vertex on the custom geometry.
17. The method as recited in claim 13 , wherein the determining the one or more spatial coordinates includes casting a 2D coordinate of each of the one or more detected features onto the template geometry using ray casting techniques.