Systems and methods for building a pseudo-muscle topology of a live actor in computer animation
An animation system wherein a machine learning model is adopted to generate animated facial actions based on parameters obtained from a live actor. Specifically, the anatomical structure such as a facial muscle topology and a skull surface that are specific to the live actor may be used. A muscle structure of simplified “pseudo” muscles that approximate the actual muscle topology but with reduced degree of freedom is determined to improve computational efficiency.
1. A computer-implemented method for building a pseudo-muscle topology of a live actor, the method comprising:
under the control of one or more computer systems configured with executable instructions:
determining, from a plurality of facial scans of the live actor, a muscle model in a form of a three-dimensional polygon topology representing a set of facial muscles of the live actor and a vector of muscle strain values corresponding to the set of facial muscles, each strain value in the vector indicating a control value for a facial muscle in the set of facial muscles;
selecting a subset of facial muscles from the set of facial muscles in the muscle model;
generating a set of pseudo-muscles that approximate the subset of facial muscles, wherein a respective pseudo-muscle from the set of pseudo-muscles is represented by a respective muscle curve;
determining a mapping between the selected subset of facial muscles and the set of pseudo-muscles;
determining, for the respective muscle curve representing the respective pseudo-muscle, a start position, an end position and a line-of-action length based on the mapping and structural parameters corresponding to the selected subset of facial muscles;
computing, for the respective pseudo-muscle, a respective pseudo-muscle strain value based on the line-of-action length and a rest length of the respective pseudo-muscle, wherein computed pseudo-muscle strain values substitute the vector of muscle strain values in the muscle model; and
sending the set of pseudo-muscles described by a set of muscle curves and corresponding pseudo-muscle strain values as a substitute of the three-dimensional polygon topology to a machine learning engine for computer animation.
2. The method of claim 1 , wherein the muscle model comprises the three-dimensional polygon topology, and each surface or edge of the three-dimensional polygon topology represents a respective facial muscle associated with a respective muscle strain value.
3. The method of claim 2 , wherein the set of pseudo-muscles have a reduced degree of freedom compared with the three-dimensional polygon topology.
4. The method of claim 1 , wherein the plurality of facial scans comprise data pertaining to the physical surface deformations, wherein the physical surface deformations are associated with facial actions.
5. The method of claim 4 , wherein the plurality of facial scans of the live actor comprises at least one facial scan of a neutral facial action of the live actor and at least one facial scan of a non-neutral facial action of the live actor.
6. The method of claim 5 , further comprising:
computing, for the respective pseudo-muscle, the rest length based on a first start position and a first end position of the respective muscle curve derived from the one facial scan of the neural facial action; and computing, for the respective pseudo-muscle, the line-of-action length based on a second start position and a second end position of the respective muscle curve derived from the one facial scan of the non-neutral facial action.
7. The method of claim 1 , further comprising selecting a different subset of facial muscles from the muscle model for pseudo-muscle generation based at least in part on performance feedback from the machine learning engine.
8. The method of claim 7 , wherein the performance feedback includes a loss comparing a mesh generated by a machine learning system based at least in part on the set of pseudo-muscles and a ground truth mesh.
9. The method of claim 7 , wherein the performance feedback includes usability feedback of a mesh generated based at least in part on the set of pseudo-muscles when the mesh is used by an animation system to animate a character.
10. The method of claim 1 , wherein the respective pseudo-muscle replaces the selected plurality of muscles in the muscle model, such that strains of the selected plurality of muscles are one or more predetermined functions of a strain on the respective pseudo-muscle.
11. A system for building a pseudo-muscle topology of a live actor, the system comprising:
a memory; and
a processor reading executable instructions from the memory to:
determine, from a plurality of facial scans of the live actor, a muscle model in a form of a three-dimensional polygon topology representing a set of facial muscles of the live actor and a vector of muscle strain values corresponding to the set of facial muscles, each strain value in the vector indicating a control value for a facial muscle in the set of facial muscles;
select a subset of facial muscles from the set of facial muscles in the muscle model;
generate a set of pseudo-muscles that approximate the subset of facial muscles, wherein a respective pseudo-muscle from the set of pseudo-muscles is represented by a respective muscle curve;
determine a mapping between the selected subset of facial muscles and the set of pseudo-muscles;
determine, for the respective muscle curve representing the respective pseudo-muscle, a start position, an end position and a line-of-action length based on the mapping and structural parameters corresponding to the selected subset of facial muscles;
compute, for the respective pseudo-muscle, a respective pseudo-muscle strain value based on the line-of-action length and a rest length of the respective pseudo-muscle, wherein computed pseudo-muscle strain values substitute the vector of muscle strain values in the muscle model; and
send the set of pseudo-muscles described by a set of muscle curves and corresponding pseudo-muscle strain values as a substitute of the three-dimensional polygon topology to a machine learning engine for computer animation.
12. The system of claim 11 , wherein the muscle model comprises the three-dimensional polygon topology, and each surface or edge of the three-dimensional polygon topology represents a respective facial muscle associated with a respective muscle strain value.
13. The system of claim 12 , wherein the set of pseudo-muscles have a reduced degree of freedom compared with the three-dimensional polygon topology.
14. The system of claim 11 , wherein the plurality of facial scans comprise data pertaining to the physical surface deformations, wherein the physical surface deformations are associated with facial actions.
15. The system of claim 14 , wherein the plurality of facial scans of the live actor comprises at least one facial scan of a neutral facial action of the live actor and at least one facial scan of a non-neutral facial action of the live actor.
16. The system of claim 15 , wherein the processor further reads executable instructions to:
compute, for the respective pseudo-muscle, the rest length based on a first start position and a first end position of the respective muscle curve derived from the one facial scan of the neural facial action; and
compute, for the respective pseudo-muscle, the line-of-action length based on a second start position and a second end position of the respective muscle curve derived from the one facial scan of the non-neutral facial action.
17. The system of claim 11 , wherein the processor further reads executable instructions to:
select a different subset of facial muscles from the muscle model for pseudo-muscle generation based at least in part on performance feedback from the machine learning engine.
18. The system of claim 17 , wherein the performance feedback includes a loss comparing a mesh generated by a machine learning system based at least in part on the set of pseudo-muscles and a ground truth mesh.
19. The system of claim 17 , wherein the performance feedback includes usability feedback of a mesh generated based at least in part on the set of pseudo-muscles when the mesh is used by an animation system to animate a character.
20. A non-transitory processor-readable medium storing a plurality of processor-executable instructions for building a pseudo-muscle topology of a live actor, the instructions being executed by a processor to:
determine, from a plurality of facial scans of the live actor, a muscle model in a form of a three-dimensional polygon topology representing a set of facial muscles of the live actor and a vector of muscle strain values corresponding to the set of facial muscles, each strain value in the vector indicating a control value for a facial muscle in the set of facial muscles;
select a subset of facial muscles from the set of facial muscles in the muscle model;
generate a set of pseudo-muscles that approximate the subset of facial muscles, wherein a respective pseudo-muscle from the set of pseudo-muscles is represented by a respective muscle curve;
determine a mapping between the selected subset of facial muscles and the set of pseudo-muscles;
determine, for the respective muscle curve representing the respective pseudo-muscle, a start position, an end position and a line-of-action length based on the mapping and structural parameters corresponding to the selected subset of facial muscles;
compute, for the respective pseudo-muscle, a respective pseudo-muscle strain value based on the line-of-action length and a rest length of the respective pseudo-muscle, wherein computed pseudo-muscle strain values substitute the vector of muscle strain values in the muscle model; and
send the set of pseudo-muscles described by a set of muscle curves and corresponding pseudo-muscle strain values as a substitute of the three-dimensional polygon topology to a machine learning engine for computer animation.