Predicting the appearance of deformable objects in video games

1. A computer-implemented method of predicting an appearance of a deformable object in a video game, comprising:

determining a configuration of a moveable object underlying the deformable object;

inputting, into a machine-learning model, input data comprising a representation of the configuration of the moveable object;

generating, as output of the machine-learning model, a model output including both a code for mesh data and a code for texture data for predicting the appearance of the deformable object;

determining, from the model output including both the code for the mesh data and the code for the texture data, the mesh and texture data for the deformable object; and

rendering the deformable object using the generated mesh and texture data.

2. The method of claim 1 , wherein determining the texture data for the deformable object comprises determining a normal map for the deformable object.

3. The method of claim 2 wherein the code for the texture data included in the model output corresponds to a code for a predicted normal map, wherein determining the normal map for the deformable object comprises performing a nearest-neighbour search on a normal map data store based on the code for the predicted normal map, and selecting the normal map for the deformable object based on a result of the nearest-neighbour search.

4. The method of claim 1 , wherein the code for the mesh data included in the model output corresponds to a code for a predicted mesh for the deformable object.

5. The method of claim 4 , wherein determining the mesh data for the deformable object comprises generating a mesh by combining stored mesh basis vectors with the code for the predicted mesh.

6. The method of claim 1 , wherein:

the moveable object comprises one or more rotatable joints;

the configuration of the moveable object comprises rotational information for the one or joints of the moveable object; and

the deformable object is a fabric that overlies the moveable object.

7. The method of claim 1 , wherein the machine-learning model comprises a neural network.

8. A computer-implemented method of training a machine-learning model for predicting an appearance of a deformable object in a video game, the method comprising:

performing one or more simulations of the deformable object overlying a moveable object, wherein the moveable object is in a particular configuration for each time step of a plurality of time steps of a simulation;

for each simulation of the one or more simulations:

generating one or more training examples for the simulation, wherein each training example comprises, for a particular time step of the simulation: (i) a representation of a configuration for the moveable object, (ii) a code for mesh data for the deformable object, and (iii) a code for texture data for the deformable object; and

updating parameters of the machine-learning model, comprising for each of one or more training examples:

inputting, into the machine-learning model, input data for the training example, the input data comprising the representation of the configuration of the moveable object;

generating, as output of the machine-learning model, a model output including both a code for a predicted normal map and a code for a predicted mesh for predicting the appearance of the deformable object; and

updating parameters of the machine-learning model based on a comparison of the model output for the training example with both: (i) the code for the mesh data, and (ii) the code for the texture data.

9. The method of claim 8 , wherein each simulation of the one or more simulations simulates a different trajectory of the moveable object starting from an initial configuration, wherein the initial configuration is the same for each of the one or more simulations.

10. The method of claim 8 , wherein generating the one or more training examples comprises:

determining mesh basis vectors for mesh data obtained from the one or more simulations;

determining texture basis vectors for texture data obtained from the one or more simulations; and

generating a training example for a time step of a simulation comprising:

determining a code for mesh data of the time step, wherein the code can be used in combination with the determined mesh basis vectors to reconstruct the mesh data for the time step; and

determining a code for texture data of the time step.

11. The method of claim 10 , wherein the basis vectors and codes are determined from performing Principal Components Analysis (PCA) on the mesh and texture data obtained from the one or more simulations.

12. The method of claim 10 , further comprising storing one or more mesh basis vectors for use in reconstructing meshes for the deformable object in the video game.

13. The method of claim 8 , further comprising:

generating a texture data store that associates texture data for a time step of a simulation with a corresponding code for the texture data.

14. The method of claim 13 , wherein the texture data of the texture data store are stored in a compressed representation.

15. The method of claim 8 , wherein:

the moveable object comprises one or more rotatable joints;

the configuration of the moveable object comprises rotational information for the one or joints of the moveable object; and

the deformable object is a fabric that overlies the moveable object.

16. A non-transitory computer-readable medium containing instructions, which when executed by one or more processors, causes the one or more processors to perform a method comprising:

determining a configuration of a moveable object underlying a deformable object in a video game;

inputting, into a machine-learning model, input data comprising a representation of the configuration of the moveable object;

generating, as output of the machine-learning model, a model output including both a code for mesh data and a code for texture data for predicting an appearance of the deformable object;

determining, from the model output including both the code for the mesh data and the code for the texture data, the mesh and texture data for the deformable object; and

rendering the deformable object using the generated mesh and texture data.

17. The non-transitory computer-readable medium of claim 16 , wherein determining the texture data for the deformable object comprises determining a normal map for the deformable object.

18. The non-transitory computer-readable medium of claim 17 , wherein the model output comprises a code for a predicted normal map, wherein determining a normal map for the deformable object comprises performing a nearest-neighbour search on a normal map data store based on the code, and selecting a normal map for the deformable object based on a result of the nearest-neighbour search.

19. The non-transitory computer-readable medium of claim 16 , wherein the model output comprises a code for a predicted mesh for the deformable object.

20. The non-transitory computer-readable medium of claim 16 , wherein determining mesh data for the deformable object comprises generating a mesh by combining stored mesh basis vectors with the code for the predicted mesh.