Systems and methods for data bundles in computer animation
An animation system wherein a machine learning model is adopted to learn a transformation relationship between facial muscle movements and skin surface movements. For example, for the skin surface representing “smile,” the transformation model derives movement vectors relating to what facial muscles are activated, what are the muscle strains, what is the joint movement, and/or the like. Such derived movement vectors may be used to simulate the skin surface “smile.”
1. A computer-implemented method for processing data derived from scans of live actors, the method comprising:
under control of one or more computer systems configured with executable instructions:
obtaining scan data from a live actor performing a facial movement over a data bundle time period;
deriving a data bundle from the scan data, wherein the data bundle comprises control values for a muscle model of the live actor for application to the muscle model and rendering as an animation, to approximate the performance, the data bundle comprising a first cache of time-varying topology data representing movement of a skin surface over the data bundle time period and a second cache of time-varying vector data representing changes to muscle strains in a muscle strain vector over the data bundle time period;
obtaining an artist movement adjustment for the skin surface relative to movement corresponding to the data bundle; and
generating a modified data bundle having a modifier first cache comprising the artist movement adjustment and a modified second cache generated by inputting the modified first cache to a machine learning model that formulates a relationship between an input skin surface and an output muscle strain vector, wherein the modified second cache, when applied to the muscle model, causes the muscle model to move in variance with the data bundle to approximate the modified first cache represented by the first cache adjusted according to the artist movement adjustment.
2. The computer-implemented method of claim 1 , wherein the scan data includes a first facial scan capturing a neutral pose of a live actor, and a second facial scan capturing a non-neutral pose of the live actor.
3. The computer-implemented method of claim 1 , wherein the data bundle further comprises a third cache of data representing any combination of jaw movements and eye movements over the data bundle time period.
4. The computer-implemented method of claim 3 , further comprising:
determining an action locus for the artist movement adjustment;
determining a range of action for the artist movement adjustment;
determining a subset of the muscle strain vector applicable to the range of action; and
limiting changes of the modified first cache relative to the first cache to changes in the subset of the muscle strain vector.
5. The computer-implemented method of claim 1 , further comprising limiting a search for values for the second cache to a manifold of allowed values for the second cache of data.
6. The computer-implemented method of claim 5 , wherein the manifold of allowed values for the second cache of data correspond to known feasible expressions of the live actor.
7. The computer-implemented method of claim 1 , wherein the facial movement is one or more of a facial action, a dialog, and/or a depiction of an emotion.
8. A computer system for processing data derived from scans of live actors, the system comprising:
one or more processors; and
a memory storing processor-executable instructions executed by the one or more processors to:
obtain scan data from a live actor performing a facial movement over a data bundle time period;
derive a data bundle from the scan data, wherein the data bundle comprises control values for a muscle model of the live actor for application to the muscle model and rendering as an animation, to approximate the performance, the data bundle comprising a first cache of data representing movement of a skin surface over the data bundle time period and a second cache of data representing changes to muscle strains in a muscle strain vector over the data bundle time period;
obtain an artist movement adjustment for the skin surface relative to movement corresponding to the data bundle; and
generate a modified data bundle having a modifier first cache comprising the artist movement adjustment and a modified second cache generated by inputting the modified first cache to a machine learning model that formulates a relationship between an input skin surface and an output muscle strain vector, wherein the modified second cache, when applied to the muscle model, causes the muscle model to move in variance with the data bundle to approximate the modified first cache represented by the first cache adjusted according to the artist movement adjustment.
9. The system of claim 8 , wherein the scan data includes a first facial scan capturing a neutral pose of a live actor, and a second facial scan capturing a non-neutral pose of the live actor.
10. The system of claim 8 , wherein the data bundle further comprises a third cache of data representing any combination of jaw movements and eye movements over the data bundle time period.
11. The system of claim 10 , wherein the one or more processors further execute the processor-executable instructions to:
determine an action locus for the artist movement adjustment;
determining a range of action for the artist movement adjustment;
determining a subset of the muscle strain vector applicable to the range of action; and
limiting changes of the modified first cache relative to the first cache to changes in the subset of the muscle strain vector.
12. The system of claim 8 , wherein the one or more processors further execute the processor-executable instructions to:
limit a search for values for the second cache to a manifold of allowed values for the second cache of data.
13. The system of claim 12 , wherein the manifold of allowed values for the second cache of data correspond to known feasible expressions of the live actor.
14. The system of claim 8 , wherein the facial movement is one or more of a facial action, a dialog, and/or a depiction of an emotion.
15. A computer-readable non-transitory storage medium storing computer-executable instructions for processing data derived from scans of live actors, the computer-executable instructions being executable by one or more processors to perform operations comprising:
obtaining scan data from a live actor performing a facial movement over a data bundle time period;
deriving a data bundle from the scan data, wherein the data bundle comprises control values for a muscle model of the live actor for application to the muscle model and rendering as an animation, to approximate the performance, the data bundle comprising a first cache of data representing movement of a skin surface over the data bundle time period and a second cache of data representing changes to muscle strains in a muscle strain vector over the data bundle time period;
obtaining an artist movement adjustment for the skin surface relative to movement corresponding to the data bundle; and
generating a modified data bundle having a modifier first cache comprising the artist movement adjustment and a modified second cache generated by inputting the modified first cache to a machine learning model that formulates a relationship between an input skin surface and an output muscle strain vector, wherein the modified second cache, when applied to the muscle model, causes the muscle model to move in variance with the data bundle to approximate the modified first cache represented by the first cache adjusted according to the artist movement adjustment.
16. The medium of claim 15 , wherein the scan data includes a first facial scan capturing a neutral pose of a live actor, and a second facial scan capturing a non-neutral pose of the live actor.
17. The medium of claim 15 , wherein the data bundle further comprises a third cache of data representing any combination of jaw movements and eye movements over the data bundle time period.
18. The medium of claim 17 , wherein the operations further comprise:
determining an action locus for the artist movement adjustment;
determining a range of action for the artist movement adjustment;
determining a subset of the muscle strain vector applicable to the range of action; and
limiting changes of the modified first cache relative to the first cache to changes in the subset of the muscle strain vector.
19. The medium of claim 15 , wherein the operations further comprise limiting a search for values for the second cache to a manifold of allowed values for the second cache of data.
20. The medium of claim 15 , wherein the manifold of allowed values for the second cache of data correspond to known feasible expressions of the live actor.