Neural network motion controller
Apparatuses, systems, and techniques to animate objects in computer-generated graphics. In at least one embodiment, one or more neural networks are trained to identify one or more forces to be applied to one or more objects based, at least in part, on training data corresponding to two or more aspects of motion of the one or more objects.
1 . A processor, comprising:
one or more circuits to cause one or more neural networks to be trained to identify one or more forces to be applied to one or more objects based, at least in part, on training data sampled according to a tree structure organizing the training data by general classes of motion at respective nodes of a first level of the tree structure and by specialized classes of motion at nodes of a second level of the tree structure branching from respective nodes of the first level.
2 . The processor of claim 1 , wherein the one or more forces are applied to one or more joints of one or more objects to cause the one or more objects to move in accordance with a physics-based simulation.
3 . The processor of claim 1 , wherein the training data is used to train the one or more neural networks in accordance with a hierarchy of the training data organized according to specialization of aspects of motion.
4 . The processor of claim 3 , wherein the one or more neural networks are trained by randomly selecting aspects of motion from the first, followed by randomly selecting aspects of motion from the second level below the first level.
5 . The processor of claim 1 , wherein an episode of training the one or more neural networks is terminated when any one of a plurality of reward terms falls below a threshold level.
6 . The processor of claim 1 , wherein a training episode is initialized to a state based on a frame of motion that is displaced by one or more frames from a starting frame of video data comprising an example of an aspect of motion.
7 . The processor of claim 1 , wherein variance of joints associated with a model of the one or more objects decays during training in accordance with a scheduled decay of variance.
8 . The processor of claim 1 , wherein the one or more neural networks comprise a motion executor to be trained to generate one or more amounts of force to apply to one or more joints of the one or more objects, based at least in part on a target state of the one or more objects supplied as input to the motion executor.
9 . A system, comprising:
one or more processors to train one or more neural networks to identify one or more forces to be applied to one or more objects based, at least in part, on training data sampled according to a tree structure organizing the training data by general classes of motion at respective nodes of a first level of the tree structure and by specialized classes of motion at nodes of a second level of the tree structure branching from respective nodes of the first level.
10 . The system of claim 9 , wherein the one or more forces are applied to one or more joints of the one or more objects.
11 . The system of claim 9 , the one or more processors to train the one or more neural networks using aspects of motion taken from the training data in accordance with a hierarchy organized according to specialization of the aspects of motion.
12 . The system of claim 11 , the one or more processors to train the one or more neural networks by randomly selecting aspects of motion from a first level of the hierarchy, followed by randomly selecting aspects of motion from a second level of the hierarchy below the first level.
13 . The system of claim 9 , the one or more processors to terminate an episode of training the one or more neural networks based, at least in part, on any one of a plurality of reward terms falling below a threshold level during the episode.
14 . The system of claim 9 , the one or more processors to initialize a training episode to a state based on a frame of motion that is displaced by one or more frames from a starting frame of an example of an aspect of motion.
15 . The system of claim 9 , wherein variance of a joint of the one or more objects decays during training in accordance with a scheduled decay of control variance.
16 . The system of claim 15 , wherein a range of control variance of a first joint of the one or more objects is initially set to an amount less than variance of a second joint.
17 . A method, comprising:
training one or more neural networks to identify one or more forces to be applied to one or more objects based, at least in part, on training data sampled according to a tree structure organizing the training data by general classes of motion at respective nodes of a first level of the tree structure and by specialized classes of motion at nodes of a second level of the tree structure branching from respective nodes of the first level.
18 . The method of claim 17 , wherein the one or more forces comprise amounts of torque to be applied to one or more joints of the one or more objects.
19 . The method of claim 17 , further comprising:
organizing the training data into a hierarchy based, at least in part, on specialization of aspects of motion demonstrated in the training data; and
training the one or more neural networks using one or more randomly selected examples of aspects of motion corresponding to a first level of the hierarchy, and subsequently training the one or more neural networks using one or more randomly selected examples of aspects of motion corresponding to a second level of the hierarchy, the second level below the first level in the hierarchy.
20 . The method of claim 17 , further comprising:
terminating an episode of training the one or more neural networks based, at least in part, on any one of a plurality of reward terms falling below a threshold level during the episode.