Enhanced animation generation based on conditional modeling
Systems and methods are provided for enhanced animation generation based on conditional modeling. An example method includes accessing an autoencoder trained based on poses and conditional information associated with the poses, each pose being defined based on location information associated with joints, and the conditional information for each pose reflecting prior poses of the pose, with the autoencoder being trained to reconstruct, via a latent variable space, each pose based on the conditional information. Poses in a sequence of poses, are obtained via an interactive user interface, and the latent variable space is sampled. An output pose is generated based on the sampling, the output pose being included in the interactive user interface.
1. A computer-implemented method comprising:
accessing an autoencoder trained based on a plurality of poses of one or more real-world persons and conditional information associated with the poses, each pose being defined based on location information associated with a plurality of joints, and the conditional information for each pose reflecting one or more prior poses of the pose, wherein the autoencoder was trained to reconstruct, via a latent variable space, each pose based on the conditional information;
obtaining, via an interactive user interface, a sequence of poses comprising a first pose and a second pose which is subsequent to the first pose, and sampling the latent variable space; and
generating, based on the autoencoder for inclusion in the interactive user interface, an output pose based on the sampling and the sequence of poses, wherein the sequence of poses is updated to include the output pose.
2. The computer-implemented method of claim 1 , wherein updating the sequence of poses comprises:
discarding the first pose from the sequence of poses; and
including the output pose in the updated sequence of poses, the output pose being subsequent to the second pose.
3. The computer-implemented method of claim 1 , further comprising:
sampling the latent variable space; and
generating a second output pose based on the sampling and the updated sequence of poses.
4. The computer-implemented method of claim 1 , wherein the output pose is blended with the sequence of poses, and wherein the blending is provided in the interactive user interface as an animation.
5. The computer-implemented method of claim 1 , wherein the autoencoder is a conditional variational autoencoder.
6. The computer-implemented method of claim 1 , wherein the latent variable space is associated with a plurality of latent variables, the latent variables reflecting Gaussian distributions, and wherein sampling the latent variable space comprises:
generating random Gaussian samples for respective values of the latent variables.
7. The computer-implemented method of claim 6 , wherein generating the output pose comprises:
providing the values of the latent variables and the sequence of poses to a decoder of the autoencoder; and
generating, by the decoder, the output pose.
8. The computer-implemented method of claim 1 , wherein each pose is further defined based on velocity information associated with the joints.
9. The computer-implemented method of claim 1 , wherein the conditional information further indicates a direction of movement, wherein a particular direction of movement is received via the interactive user interface, and wherein the output pose is generated based on the particular direction of movement.
10. Non-transitory computer storage media storing instructions that when executed by a system of one or more computers, cause the one or more computers to perform operations comprising:
accessing an autoencoder trained based on a plurality of poses and conditional information associated with the poses, each pose being defined based on location information associated with a plurality of joints, and the conditional information for each pose reflecting one or more prior poses of the pose, wherein the autoencoder was trained to reconstruct, via a latent variable space, each pose based on the conditional information;
obtaining, via an interactive user interface, a sequence of poses comprising a first pose and a second pose which is subsequent to the first pose, and sampling the latent variable space; and
generating, based on the autoencoder for inclusion in the interactive user interface, an output pose based on the sampling and the sequence of poses, wherein the sequence of poses is updated to include the output pose.
11. The non-transitory computer storage media of claim 10 , wherein updating the sequence of poses comprises:
discarding the first pose from the sequence of poses; and
including the output pose in the updated sequence of poses, the output pose being subsequent to the second pose.
12. The non-transitory computer storage media of claim 10 , wherein the operations further comprise:
sampling the latent variable space; and
generating a second output pose based on the sampling and the updated sequence of poses.
13. The non-transitory computer storage media of claim 10 , wherein the output pose is blended with the sequence of poses, and wherein the blending is provided in the interactive user interface as an animation.
14. The non-transitory computer storage media of claim 10 , wherein the latent variable space is associated with a plurality of latent variables, the latent variables reflecting Gaussian distributions, and wherein sampling the latent variable space comprises:
generating random Gaussian samples for respective values of the latent variables,
wherein the values of the latent variables and the sequence of poses are provided as an input to a decoder of the autoencoder, and wherein the decoder generates the output pose.
15. The non-transitory computer storage media of claim 10 , wherein the conditional information further indicates a direction of movement, wherein a particular direction of movement is received via the interactive user interface, and wherein the output pose is generated based on the particular direction of movement.
16. A system comprising one or more computers and non-transitory computer storage media storing instructions that when executed by the one or more computers, cause the one or more computers to perform operations comprising:
accessing an autoencoder trained based on a plurality of poses and conditional information associated with the poses, each pose being defined based on location information associated with a plurality of joints, and the conditional information for each pose reflecting one or more prior poses of the pose, wherein the autoencoder was trained to reconstruct, via a latent variable space, each pose based on the conditional information;
obtaining, via an interactive user interface, a sequence of poses comprising a first pose and a second pose which is subsequent to the first pose, and sampling the latent variable space; and
generating, based on the autoencoder for inclusion in the interactive user interface, an output pose based on the sampling and the sequence of poses, wherein the sequence of poses is updated to include the output pose.
17. The system of claim 16 , wherein the operations further comprise:
sampling the latent variable space; and
generating a second output pose based on the sampling and the updated sequence of poses.
18. The system of claim 16 , wherein the output pose is blended with the sequence of poses, and wherein the blending is provided in the interactive user interface as an animation.
19. The system of claim 16 , wherein the conditional information further indicates a direction of movement, wherein a particular direction of movement is received via the interactive user interface, and wherein the output pose is generated based on the particular direction of movement.
20. The system of claim 16 , wherein the sequence of poses is provided as input to the autoencoder.