Systems and methods for building a pseudo-muscle topology of a live actor in computer animation
An animation system wherein a machine learning model is adopted to generate animated facial actions based on parameters obtained from a live actor. Specifically, the anatomical structure such as a facial muscle topology and a skull surface that are specific to the live actor may be used. A muscle structure of simplified “pseudo” muscles that approximate the actual muscle topology but with reduced degree of freedom is determined to improve computational efficiency.
1. A computer-implemented method for building a pseudo-muscle topology of a live actor, the method comprising:
under the control of one or more computer systems configured with executable instructions:
determining, from a plurality of facial scans of the live actor, a muscle model representing facial muscles of the live actor and a vector of muscle strain values;
select a subset of facial muscles from the muscle model;
generating a set of pseudo-muscles that approximate the subset of facial muscles, wherein a respective pseudo-muscle of the set of pseudo-muscles is represented by a respective muscle curve that is a line connecting a start position and an end position;
determining a mapping between the selected plurality of muscles and the set of pseudo-muscles;
determining, for the respective muscle curve representing the respective pseudo-muscle, the start position, the end position and a line-of-action length based on the mapping;
computing, for the respective pseudo-muscle, a respective pseudo-muscle strain value based on the line-of-action length of the respective pseudo-muscle and a rest length of the respective pseudo-muscle; and
training a machine learning engine for animating a facial expression based on training facial muscle data described by the set of pseudo-muscles described by a set of muscle curves and corresponding pseudo-muscle strain values that replaces the vector of muscle strain values.
2. The method of claim 1 , wherein the muscle model comprises a three-dimensional polygon topology, and each surface or edge of the three-dimensional polygon topology represents a respective facial muscle associated with a respective muscle strain value.
3. The method of claim 2 , wherein the set of pseudo-muscles have a reduced degree of freedom compared with the three-dimensional polygon topology.
4. The method of claim 1 , wherein the plurality of facial scans comprise data pertaining to the physical surface deformations, wherein the physical surface deformations are associated with facial actions.
5. The method of claim 4 , wherein the plurality of facial scans of the live actor comprises at least one facial scan of a neutral facial action of the live actor and at least one facial scan of a non-neutral facial action of the live actor.
6. The method of claim 5 , further comprising:
computing, for the respective pseudo-muscle, the rest length based on a first start position and a first end position of the respective muscle curve derived from the one facial scan of the neural facial action; and computing, for the respective pseudo-muscle, the line-of-action length based on a second start position and a second end position of the respective muscle curve derived from the one facial scan of the non-neutral facial action.
7. The method of claim 1 , further comprising selecting a different subset of facial muscles from the muscle model for pseudo-muscle generation based at least in part on performance feedback from the machine learning engine.
8. The method of claim 7 , wherein the performance feedback includes a loss, wherein the loss is determined by comparing a mesh generated by a machine learning system based at least in part on the set of pseudo-muscles and a ground truth mesh.
9. The method of claim 7 , wherein the performance feedback includes usability feedback of a mesh generated based at least in part on the set of pseudo-muscles when the mesh is used by an animation system to animate a character.
10. The method of claim 1 , wherein the respective pseudo-muscle replaces the selected plurality of muscles in the muscle model, such that strains of the selected plurality of muscles are one or more predetermined functions of a strain on the respective pseudo-muscle.
11. A system for building a pseudo-muscle topology of a live actor, the system comprising:
a memory; and
a processor reading executable instructions from the memory to:
determine, from a plurality of facial scans of the live actor, a muscle model representing facial muscles of the live actor and a vector of muscle strain values;
select a subset of facial muscles from the muscle model;
generate a set of pseudo-muscles that approximate the subset of facial muscles, wherein a respective pseudo-muscle of the set of pseudo-muscles is represented by a respective muscle curve that is a line connecting a start position and an end position;
determine a mapping between the selected plurality of muscles and the set of pseudo-muscles;
determine, for the respective muscle curve representing the respective pseudo-muscle, the start position, the end position and a line-of-action length based on the mapping;
compute, for the respective pseudo-muscle, a respective pseudo-muscle strain value based on the line-of-action length of the respective pseudo-muscle and a rest length of the respective pseudo-muscle; and
train a machine learning engine for animating a facial expression based on training facial muscle data described by the set of pseudo-muscles described by a set of muscle curves and corresponding pseudo-muscle strain values that replaces the vector of muscle strain values.
12. The system of claim 11 , wherein the muscle model comprises a three- dimensional polygon topology, and each surface or edge of the three-dimensional polygon topology represents a respective facial muscle associated with a respective muscle strain value.
13. The system of claim 12 , wherein the set of pseudo-muscles have a reduced degree of freedom compared with the three-dimensional polygon topology.
14. The system of claim 11 , wherein the plurality of facial scans comprise data pertaining to the physical surface deformations, wherein the physical surface deformations are associated with facial actions.
15. The system of claim 14 , wherein the plurality of facial scans of the live actor comprises at least one facial scan of a neutral facial action of the live actor and at least one facial scan of a non-neutral facial action of the live actor.
16. The system of claim 15 , wherein the processor further reads executable instructions to:
compute, for the respective pseudo-muscle, the rest length based on a first start position and a first end position of the respective muscle curve derived from the one facial scan of the neural facial action; and
compute, for the respective pseudo-muscle, the line-of-action length based on a second start position and a second end position of the respective muscle curve derived from the one facial scan of the non-neutral facial action.
17. The system of claim 11 , wherein the processor further reads executable instructions to:
select a different subset of facial muscles from the muscle model for pseudo-muscle generation based at least in part on performance feedback from the machine learning engine.
18. The system of claim 17 , wherein the performance feedback includes a loss, wherein the loss is determined by comparing a mesh generated by a machine learning system based at least in part on the set of pseudo-muscles and a ground truth mesh.
19. The system of claim 17 , wherein the performance feedback includes usability feedback of a mesh generated based at least in part on the set of pseudo-muscles when the mesh is used by an animation system to animate a character.
20. The system of claim 11 , wherein the respective pseudo-muscle replaces the selected plurality of muscles in the muscle model, such that strains of the selected plurality of muscles are one or more predetermined functions of a strain on the respective pseudo-muscle.