Systems and methods for generating a skull surface for computer animation
An animation system wherein a machine learning model is adopted to generate animated facial actions based on parameters obtained from a live actor. Specifically, the anatomical structure such as a facial muscle topology and a skull surface that are specific to the live actor may be used. A skull surface that is specific to a live actor based on facial scans of the live actor and generic tissue depth data. For example, the facial scans of the live actor may provide a skin surface topology of the live actor, based on which the skull surface underneath the skin surface can be derived by “offsetting” the skin surface with corresponding soft tissue depth at different sampled points on the skin surface.
1. A computer-implemented method for generating a bone surface depicting at least a portion of a skull of a live actor, the method comprising:
obtaining a plurality of facial scans of the live actor, each facial scan including a respective skin surface and respective sensing data indicative of facial muscle strains corresponding to a set of facial muscles;
obtaining, from a database, a tissue depth dataset including a tissue depth indicating a distance from a corresponding surface point on the human face to a skull surface underneath the human face;
deriving a bone surface based on a skin surface generated from the plurality of facial scans and the tissue depth dataset;
refining the bone surface based on a user input via a user interface; and
refining the bone surface via a neural network by inputting to the neural network the plurality of facial scans and a tissue depth array from the tissue depth dataset, and using muscle or skin parameters derived from the plurality of facial scans as ground truth labels.
2. The method of claim 1 , further comprising:
determining, from the plurality of facial scans, the three-dimensional facial skin surface structure of the live actor by:
determining, for a sampled point on the three-dimensional facial skin surface structure, a corresponding point on the bone surface based on a respective tissue depth corresponding to the sampled point; and
determining the bone surface by interpolating a plurality of determined corresponding points on the bone surface.
3. The method of claim 1 , further comprising:
generating a muscle model from the plurality of the facial scans of the live actor, wherein the muscle model comprises a plurality of facial muscles and each facial muscle includes an attachment point to the skull; and
refining the bone surface according to a set of attachment points indicated by the generated muscle model.
4. The method of claim 1 , further comprising:
receiving, via the user interface, the user input to modify the bone surface; and
modifying the bone surface according to the user input.
5. The method of claim 1 , wherein the tissue depth dataset is selected from a plurality of tissue depth datasets based on a known characteristic of the live actor.
6. The method of claim 5 , wherein each dataset of the plurality of tissue depth datasets corresponds to any combination of following factors:
race;
age;
gender;
height;
weight; and
body mass index.
7. A system for generating a bone surface depicting at least a portion of a skull of a live actor, the system comprising:
an input interface configured to obtain a plurality of facial scans of the live actor, each facial scan including a respective skin surface and respective sensing data indicative of facial muscle strains corresponding to a set of facial muscles;
a communication interface configured to obtain, from a database, a tissue depth dataset including a tissue depth indicating a distance from a corresponding surface point on the human face to a skull surface underneath the human face;
one or more processors configured to:
derive a bone surface based on a skin surface generated from the plurality of facial scans and the tissue depth dataset;
refine the bone surface based on a user input via a user interface; and
refine the bone surface via a neural network by inputting to the neural network the plurality of facial scans and a tissue depth array from the tissue depth dataset, and using muscle or skin parameters derived from the plurality of facial scans as ground truth labels.
8. The system of claim 7 , wherein the one or more processors are further configured to determine, from the plurality of facial scans, the three-dimensional facial skin surface structure of the live actor by:
determining, for a sampled point on the three-dimensional facial skin surface structure, a corresponding point on the bone surface based on a respective tissue depth corresponding to the sampled point;
determining the bone surface by interpolating a plurality of determined corresponding points on the bone surface.
9. The system of claim 7 , wherein the one or more processors are further configured to:
generate a muscle model from the plurality of the facial scans of the live actor, wherein the muscle model comprises a plurality of facial muscles and each facial muscle includes an attachment point to the skull; and
refine the bone surface according to a set of attachment points indicated by the generated muscle model.
10. The system of claim 7 , wherein the one or more processors are further configured to:
receive, via the user interface, the user input to modify the bone surface; and
modify the bone surface according to the user input.
11. The system of claim 8 , wherein the tissue depth dataset is selected from a plurality of tissue depth datasets based on a known characteristic of the live actor.
12. The system of claim 11 , wherein each dataset of the plurality of tissue depth datasets corresponds to any combination of following factors:
race;
age;
gender;
height;
weight; and
body mass index.
13. A computer-readable non-transitory medium storing computer-executable instructions for generating a bone surface depicting at least a portion of a skull of a live actor, the computer-executable instructions being executed by one or more processors to perform operations comprising:
obtaining a plurality of facial scans of the live actor, each facial scan including a respective skin surface and respective sensing data indicative of facial muscle strains corresponding to a set of facial muscles;
obtaining, from a database, a tissue depth dataset including a tissue depth indicating a distance from a corresponding surface point on the human face to a skull surface underneath the human face;
deriving a bone surface based on a skin surface generated from the plurality of facial scans and the tissue depth dataset;
refining the bone surface based on a user input via a user interface; and
refining the bone surface via a neural network by inputting to the neural network the plurality of facial scans and a tissue depth array from the tissue depth dataset, and using muscle or skin parameters derived from the plurality of facial scans as ground truth labels.
14. The computer-readable non-transitory medium of claim 13 , further comprising an operation of determining, from the plurality of facial scans, the three-dimensional facial skin surface structure of the live actor that comprises:
determining, for a sampled point on the three-dimensional facial skin surface structure, a corresponding point on the bone surface based on a respective tissue depth corresponding to the sampled point;
determining the bone surface by interpolating a plurality of determined corresponding points on the bone surface.
15. The computer-readable non-transitory medium of claim 13 , wherein the operations further comprise:
generating a muscle model from the plurality of the facial scans of the live actor, wherein the muscle model comprises a plurality of facial muscles and each facial muscle includes an attachment point to the skull; and
refining the bone surface according to a set of attachment points indicated by the generated muscle model.
16. The computer-readable non-transitory medium of claim 13 , wherein the tissue depth dataset is selected from a plurality of tissue depth datasets based on a known characteristic of the live actor.
17. The computer-readable non-transitory medium of claim 13 , wherein each dataset of the plurality of tissue depth datasets corresponds to any combination of following factors:
race;
age;
gender;
height;
weight; and
body mass index.