Body shape and pose estimation via volumetric regressor for raw three dimensional scan models

1. An electronic device for object rigging, the electronic device comprising:

a processor configured to:

obtain a three-dimensional (3D) scan of an object;

identify 3D coordinates associated with joints of the 3D scan;

identify parameters associated with fitting a 3D parametric body model to the 3D scan based on the 3D coordinates of the joints, wherein, to identify the parameters, the processor is further configured to identify a bone length connecting two joints of the 3D parametric body model based on a distance between the two joints of the 3D scan and a kinematic tree layer;

modify the parameters to reduce 3D joint errors between the 3D coordinates associated with the joints on the 3D scan and the 3D parametric body model; and

generate a rigged 3D scan based on the modified parameters, for performing an animated motion.

2. The electronic device of claim 1 , wherein:

the processor is further configured to:

convert the 3D scan to a volumetric representation, based on a 3D grid of a predefined size, and

identify, from the converted 3D scan, the 3D coordinates associated with the joints; and

to identify the parameters, the processor is configured to map the 3D coordinates from the converted 3D scan onto the 3D parametric body model.

3. The electronic device of claim 1 , wherein a neural network is used to identify the joints of the 3D scan.

4. The electronic device of claim 3 , wherein the neural network includes a first neural network that is used when the 3D scan corresponds to a male and a second neural network that is used when the 3D scan corresponds to a female.

5. The electronic device of claim 1 wherein to identify the parameters, the processor is further configured to:

identify a normalized bone direction between the two joints based on a direction between the two joints and the kinematic tree layer.

6. The electronic device of claim 5 wherein the processor is further configured to:

compare the normalized bone direction to a reference bone direction; and

compare the bone length to a reference bone length.

7. The electronic device of claim 6 , wherein the parameters include:

pose parameters corresponding to a pose of the 3D scan, and

shape parameters corresponding to a shape of the 3D scan,

wherein the pose parameters and the shape parameters are based on the normalized bone direction, the bone length, the reference bone direction and the reference bone length.

8. The electronic device of claim 1 , wherein to identify the joints, the processor is further configured to generate a 3D heatmap, a value associated with the 3D heatmap indicating a probability of a location of one joint.

9. The electronic device of claim 1 , wherein to reduce the 3D joint errors, the processor is configured to:

identify the 3D joint errors as distances between the 3D coordinates associated with the joints and joint coordinates of the 3D parametric body model; and

reduce the distances over multiple iterations.

10. The electronic device of claim 1 , wherein the 3D scan is a 3D mesh.

11. A method for object rigging, comprising:

obtaining a three-dimensional (3D) scan of an object;

identifying 3D coordinates associated with joints of the 3D scan;

identifying parameters associated with fitting a 3D parametric body model to the 3D scan based on the 3D coordinates of the joints, wherein identifying the parameters comprises identifying a bone length connecting two joints of the 3D parametric body model based on a distance between the two joints of the 3D scan and a kinematic tree layer;

modifying the parameters to reduce 3D joint errors between the 3D coordinates associated with the joints on the 3D scan and the 3D parametric body model; and

generating a rigged 3D scan based on the modified parameters, for performing an animated motion.

12. The method of claim 11 , further comprising:

converting the 3D scan to a volumetric representation, based on a 3D grid of a predefined size, and

identifying, from the converted 3D scan, the 3D coordinates associated with the joints; and

wherein identifying the parameters, comprises mapping the 3D coordinates from the converted 3D scan onto the 3D parametric body model.

13. The method of claim 11 , wherein a neural network is used to identify the joints of the 3D scan.

14. The method of claim 13 , wherein the neural network includes a first neural network that is used when the 3D scan corresponds to a male and a second neural network that is used when the 3D scan corresponds to a female.

15. The method of claim 11 wherein identifying the parameters, comprises:

identifying a normalized bone direction between the two joints based on a direction between the two joints and the kinematic tree layer.

16. The method of claim 15 further comprising:

comparing the normalized bone direction to a reference bone direction; and

comparing the bone length to a reference bone length.

17. The method of claim 16 , wherein the parameters include:

pose parameters corresponding to a pose of the 3D scan, and

shape parameters corresponding to a shape of the 3D scan,

wherein the pose parameters and the shape parameters are based on the normalized bone direction, the bone length, the reference bone direction and the reference bone length.

18. The method of claim 11 , wherein identifying the joints comprises generating a 3D heatmap, a value associated with the 3D heatmap indicating a probability of a location of one joint.

19. The method of claim 11 , wherein reducing the 3D joint errors comprises:

identifying the 3D joint errors as distances between the 3D coordinates associated with the joints and joint coordinates of the 3D parametric body model; and

reducing the distances over multiple iterations.

20. The method of claim 11 , wherein the 3D scan is a 3D mesh.