Stable spaces for rendering character garments in real-time
Techniques are disclosed for providing a learning-based clothing model that enables the simultaneous animation of multiple detailed garments in real-time. A simple conditional model learns and preserves key dynamic properties of cloth motions and folding details. Such a conditional model may be generated for each garment worn by a given character. Once generated, the conditional model may be used to determine complex body/cloth interactions in order to render the character and garment from frame-to-frame. The clothing model may be used for a variety of garments worn by male and female human characters (as well as non-human characters) while performing a varied set of motions typically used in video games (e.g., walking, running, jumping, turning, etc.).
1. A method for generating a cloth behavior model, comprising:
generating a first low-dimensional representation, wherein the first low-dimensional representation models a cloth in contact with an object;
generating a second low-dimensional representation, wherein the second low-dimensional representation models a surface of the object in a linear cloth space, wherein the surface contacts the cloth;
generating, as the cloth behavior model and on the basis of the first and second low-dimensional representations, a dynamical model of the cloth in the linear cloth space, wherein the cloth behavior model comprises:
a conditional kinematic model configured to represent motions of the cloth without a dynamic component,
a latent linear dynamical system model to model dynamic behavior of the cloth in contact with the object, and
a residual dynamics model; and
storing the resulting cloth behavior model for use in animating the cloth in contact with the object.
2. The method of claim 1 , wherein the first and second low-dimensional representations encapsulate a set of latent deformations of the cloth and the object respectively.
3. The method of claim 1 , wherein the object is a character and the cloth in contact with the object is a garment worn by the character.
4. The method of claim 3 , further comprising:
generating a sequence of animation poses of the character wearing the garment, wherein the state of the garment relative to the character is determined in each frame according to the cloth behavior model.
5. The method of claim 4 , further comprising:
computing regions of the character which are occluded by the garment; and while generating the sequence of animation poses, discarding one or more rendering fragments of the character that are occluded by the garment.
6. The method of claim 1 , wherein the cloth behavior model is a linear dynamical system (LDS) model, and further comprising:
predicting a current state of the cloth by parameterizing the cloth behavior model on one or more previous states of the cloth and the relative position and heading of the object; and
based on the predicted state of the cloth, rendering an animation frame depicting the cloth in contact with the object.
7. The method of claim 1 , wherein the first and second low-dimensional representations are generated using principal component analysis.
8. The method of claim 1 , wherein the object is a character and the cloth in contact with the object is a garment worn by the character, and wherein the first and second low-dimensional representations are generated from motion capture data representing a choreography of motions to be performed by the character.
9. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause a computer system to perform an operation for generating a cloth behavior model, the operation comprising:
generating a first low-dimensional representation modeling a cloth in contact with an object;
generating a second low-dimensional representation, wherein the second low-dimensional representation models a surface of the object in a linear cloth space, wherein the surface contacts the cloth;
generating, as the cloth behavior model and on the basis of the first and second low-dimensional representations, a dynamical model of the cloth in the linear cloth space, wherein the cloth behavior model comprises:
a conditional kinematic model configured to represent motions of the cloth without a dynamic component,
a latent linear dynamical system model to model dynamic behavior of the cloth in contact with the object, and
a residual dynamics model; and
storing the resulting cloth behavior model for use in animating the cloth in contact with the object.
10. The non-transitory computer-readable storage medium of claim 9 , wherein the first and second low-dimensional representations encapsulate a set of latent deformations of the cloth and the object respectively.
11. The non-transitory computer-readable storage medium of claim 9 , wherein the object is a character and the cloth in contact with the object is a garment worn by the character.
12. The non-transitory computer-readable storage medium of claim 11 , wherein the operation further comprises:
generating a sequence of animation poses of the character wearing the garment, wherein the state of the garment relative to the character is determined in each frame according to the cloth behavior model.
13. The non-transitory computer-readable storage medium of claim 12 , wherein the operation further comprises:
computing regions of the character which are occluded by the garment; and while generating the sequence of animation poses, discarding one or more rendering fragments of the character that are occluded by the garment.
14. The non-transitory computer-readable storage medium of claim 9 , wherein the cloth behavior model is a linear dynamical system (LDS) model, and wherein the operation further comprises:
predicting a current state of the cloth by parameterizing the cloth behavior model on one or more previous states of the cloth and the relative position and heading of the object; and
based on the predicted state of the cloth, rendering an animation frame depicting the cloth in contact with the object.
15. The non-transitory computer-readable storage medium of claim 9 , wherein the first and second low-dimensional representations are generated using principal component analysis.
16. The non-transitory computer-readable storage medium of claim 9 , wherein the object is a character and the cloth in contact with the object is a garment worn by the character and wherein the first and second low-dimensional representations are generated from motion capture data representing a choreography of motions to be performed by the character.
17. A system, comprising:
a processor; and
a memory configured to perform an operation for generating a clothing model, the operation comprising:
generating a first low-dimensional representation modeling a cloth in contact with an object;
generating a second low-dimensional representation, wherein the second low-dimensional representation models a surface of the object in a linear cloth space, wherein the surface contacts the cloth;
generating, as the cloth behavior model and on the basis of the first and second low-dimensional representations, a dynamical model of the cloth in the linear cloth space, wherein the cloth behavior model comprises:
a conditional kinematic model configured to represent motions of the cloth without a dynamic component,
a latent linear dynamical system model to model dynamic behavior of the cloth in contact with the object, and
a residual dynamics model; and
storing the resulting cloth behavior model for use in animating the cloth in contact with the object.
18. The system of claim 17 , wherein the first and second low-dimensional representations encapsulate a set of latent deformations of the cloth and the object respectively.
19. The system of claim 17 , wherein the object is a character and the cloth in contact with the object is a garment worn by the character.
20. The system of claim 19 , wherein the cloth behavior model is used to generate a sequence of animation poses of the character wearing the garment, wherein the state of the garment relative to the character is determined in each pose according to the cloth behavior model.
21. The system of claim 20 , wherein the cloth behavior model is further used to compute regions of the character occluded by the garment; and when used to generate the sequence of animation poses, discard one or more rendering fragments of the character that are occluded by the garment.
22. The system of claim 17 , wherein the cloth behavior model is a linear dynamical system (LDS) model, and wherein the operation further comprises:
predicting a current state of the cloth by parameterizing the cloth behavior model on one or more previous states of the cloth and the relative position and heading of the object; and
based on the predicted state of the cloth, rendering an animation frame depicting the cloth in contact with the object.
23. The system of claim 17 , wherein the first and second low-dimensional representations are generated using principal component analysis.
24. The system of claim 17 , wherein the object is a character and the cloth in contact with the object is a garment worn by the character and wherein the first and second low-dimensional representations are generated from motion capture data representing a choreography of motions to be performed by the character.