Enhanced pose generation based on conditional modeling of inverse kinematics
Systems and methods are provided for enhanced pose generation based on conditional modeling of inverse kinematics. An example method includes accessing an autoencoder trained based on poses, with each pose being defined based on location information of joints, and the autoencoder being trained based on conditional information indicating positions of a subset of the joints. The autoencoder is trained to reconstruct, via a latent variable space, each pose based on the conditional information. Information specifying positions of the subset of the joints is obtained via an interactive user interface and the latent variable space is sampled. An output is generated for inclusion in the interactive user interface based on the sampling and the positions.
1. A computer-implemented method comprising:
accessing an autoencoder trained based on a plurality of poses of one or more real-world persons and conditional information associated with the poses, wherein the conditional information for each pose reflects one or more prior poses of the pose;
obtaining a sequence of poses comprising a first pose and a second pose which is subsequent to the first pose; and
generating, based on the autoencoder for inclusion in an interactive user interface, an output pose based on the sequence of poses, wherein the sequence of poses is updated to include the output pose.
2. The method of claim 1 , wherein the sequence of poses is provided via the interactive user interface.
3. The method of claim 1 , wherein the updated sequence of poses is blended to form an animation.
4. The method of claim 1 , wherein the sequence of poses is obtained from an animation of a character or from video of a real-world person.
5. The method of claim 1 , wherein the conditional information further includes position information associated with a skeleton.
6. The method of claim 1 , wherein the first pose is discarded from the updated sequence of poses.
7. The method of claim 1 , wherein the autoencoder was trained to reconstruct, via a latent variable space, each pose based on the conditional information.
8. The method of claim 7 , further comprising:
sampling the latent variable space; and
generating a second output pose based on the sampling and the updated sequence of poses.
9. The method of claim 7 , wherein the latent variable space is associated with a plurality of latent variables, the latent variables reflecting Gaussian distributions, and wherein sampling the latent variable space comprises:
generating random Gaussian samples for respective values of the latent variables.
10. The method of claim 1 , wherein each pose is further defined based on velocity information associated with joints.
11. Non-transitory computer storage media storing instructions that when executed by a system of one or more computers, cause the one or more computers to perform operations comprising:
accessing an autoencoder trained based on a plurality of poses of one or more real-world persons and conditional information associated with the poses, wherein the conditional information for each pose reflects one or more prior poses of the pose;
obtaining a sequence of poses comprising a first pose and a second pose which is subsequent to the first pose; and
generating, based on the autoencoder for inclusion in an interactive user interface, an output pose based on the sequence of poses, wherein the sequence of poses is updated to include the output pose.
12. The computer storage media of claim 11 , wherein the sequence of poses is provided via the interactive user interface.
13. The computer storage media of claim 11 , wherein the updated sequence of poses is blended to form an animation which is configured for inclusion in the interactive user interface.
14. The computer storage media of claim 11 , wherein the conditional information further includes position information associated with a skeleton.
15. The computer storage media of claim 11 , wherein the autoencoder was trained to reconstruct, via a latent variable space, each pose based on the conditional information.
16. The computer storage media of claim 15 , wherein the operations further comprise:
sampling the latent variable space; and
generating a second output pose based on the sampling and the updated sequence of poses.
17. The computer storage media of claim 11 , wherein each pose is further defined based on velocity information associated with joints.
18. A system comprising one or more computers and non-transitory computer storage media storing instructions that when executed by the one or more computers, cause the one or more computers to perform operations comprising:
accessing an autoencoder trained based on a plurality of poses of one or more real-world persons and conditional information associated with the poses, wherein the conditional information for each pose reflects one or more prior poses of the pose;
obtaining a sequence of poses comprising a first pose and a second pose which is subsequent to the first pose; and
generating, based on the autoencoder for inclusion in an interactive user interface, an output pose based on the sequence of poses, wherein the sequence of poses is updated to include the output pose.
19. The system of claim 18 , wherein the updated sequence of poses is blended to form an animation which is configured for inclusion in the interactive user interface.
20. The system of claim 18 , wherein each pose is further defined based on velocity information associated with joints.