Skeletal systems for animating virtual avatars
Skinning parameters used to animate a virtual avatar can include mesh weights and joint transforms of a skeleton. Systems and methods are provided for determining skinning parameters using an optimization process subject to constraints based on human-understandable or anatomically-motivated relationships among skeletal joints. Input to the optimization process can include a high-order skeleton and the applied constraints can dynamically change during the optimization. The skinning parameters can be used in linear blend skinning (LBS) applications in augmented reality.
1. A system for automatically painting weights for linear blend skinning and determining bone locations, the system comprising:
a hardware processor in communication with a non-transitory storage, the hardware processor programmed to:
solve an error minimization equation based on an initialization skeleton in an initial pose and a target pose to derive skinning parameters that provide a match between initialization mesh vertices of an initialization mesh that corresponds to the initial pose and target mesh vertices of a target mesh that corresponds to the target pose, wherein the error minimization equation is subject to a set of constraints, and wherein derivation of the skinning parameters comprises determining using the error minimization equation that a difference between 1) positions of the initialization mesh vertices adjusted as a result of a transformation of the initialization skeleton into the target pose and 2) positions of a corresponding set of one or more target mesh vertices satisfies an error threshold; and
output the skinning parameters.
2. The system of claim 1 , wherein the skinning parameters comprise one or more weights, one or more rotations, and one or more translations associated with transformation of one or more joints of the initialization skeleton to a different pose.
3. The system of claim 1 , wherein one or more joints of the initialization skeleton are initially transformed using an initial linear blend skinning weight set and subsequently transformed by a refined linear blend skinning weight set determined by solving the error minimization equation.
4. The system of claim 3 , wherein the hardware processor is programmed to iteratively solve the error minimization equation.
5. The system of claim 1 , wherein the initialization skeleton comprises a high-order skeleton.
6. The system of claim 1 , wherein the set of constraints comprises a first constraint that restricts a child joint to being a fixed distance from a parent joint, a second constraint that a joint can translate only along a specific axis, and a third constraint that sets a weight for a vertex of the initialization skeleton to zero if a joint is located beyond a threshold distance from the vertex.
7. The system of claim 6 , wherein the set of constraints comprises a constraint that a restricted pose space for a limb rotation is based on an anatomical constraint.
8. The system of claim 7 , wherein the limb rotation comprises an arm that is restricted to not move through a digital character's torso.
9. The system of claim 6 , wherein:
the set of constraints further comprises a constraint incorporating a segmentation set that relates a set of one or more initialization mesh vertices to a set of one or more joints of the initialization skeleton.
10. The system of claim 6 , wherein the set of constraints further comprises a constraint that locks a subset of joints from being updated when solving the error minimization equation.
11. The system of claim 1 , wherein the set of constraints comprises a non-rigid constraint.
12. A method for automatically painting weights for linear blend skinning and determining bone locations comprising:
under control of a hardware processor in communication with a non-transitory storage:
solving an error minimization equation based on an initialization skeleton in an initial pose and a target pose to derive skinning parameters that provide a match between initialization mesh vertices of an initialization mesh that corresponds to the initial pose and target mesh vertices of a target mesh that corresponds to the target pose, wherein the error minimization equation is subject to a set of constraints, and wherein derivation of the skinning parameters comprises determining using the error minimization equation that a difference between 1) positions of the initialization mesh vertices adjusted as a result of a transformation of the initialization skeleton into the target pose and 2) positions of a corresponding set of one or more target mesh vertices satisfies an error threshold; and
outputting the skinning parameters.
13. The method of claim 12 , wherein the skinning parameters comprise one or more weights, one or more rotations, and one or more translations associated with transformation of one or more joints of the initialization skeleton to a different pose.
14. The method of claim 12 , wherein one or more joints of the initialization skeleton are initially transformed using an initial linear blend skinning weight set and subsequently transformed by a refined linear blend skinning weight set determined by solving the error minimization equation.
15. The method of claim 14 , wherein the method comprises iteratively solving the error minimization equation.
16. The method of claim 12 , wherein the initialization skeleton comprises a high-order skeleton.
17. The method of claim 12 , wherein the set of constraints comprises a first constraint that restricts a child joint to being a fixed distance from a parent joint, a second constraint that a joint can translate only along a specific axis, and a third constraint that sets a weight for a vertex of the initialization skeleton to zero if a joint is located beyond a threshold distance from the vertex.
18. The method of claim 17 , wherein the set of constraints further comprises a constraint that a restricted pose space for a limb rotation is based on an anatomical constraint.
19. The method of claim 18 , wherein the limb rotation comprises an arm that is restricted to not move through a digital character's torso.
20. The method of claim 17 , wherein:
the set of constraints further comprises a constraint incorporating a segmentation set that relates a set of one or more initialization mesh vertices to a set of one or more joints of the initialization skeleton.
21. The method of claim 12 , wherein the set of constraints comprises a non-rigid constraint.