Using three-dimensional scans of a physical subject to determine positions and/or orientations of skeletal joints in the rigging for a virtual character
Systems and methods for using three-dimensional scans of a physical subject to determine positions and/or orientations of skeletal joints in the rigging for a virtual character. At least one articulation segment of a polygon mesh for the virtual character may be determined. The articulation segment may include a subset of vertices in the polygon mesh. An indicator of the position or orientation of the articulation segment of the polygon mesh may be determined. Based on the indicator of the position or orientation of the articulation segment, the position or orientation of at least one joint for deforming the polygon mesh may be determined.
1. A method comprising:
receiving a plurality of polygon meshes of a digital character, the polygon meshes being representative of a physical subject in a plurality of training poses, each training pose comprising a unique articulation of one or more body parts of the physical subject;
segmenting each of the plurality of polygon meshes corresponding to the plurality of training poses into a plurality of articulation segments;
determining an indicator of position or orientation for each of the articulation segments for each of the polygon meshes corresponding to the plurality of training poses;
determining, based on the indicator of position or orientation of the each of the articulation segments, the position or orientation of one or more joints for the plurality of training poses; and
creating, based on the position or orientation of the one or more joints for the plurality of training poses, a pose space for a pose space deformer to calculate mesh deformations for the digital character by interpolating between the plurality of training poses,
wherein determining an indicator of position or orientation for each of the articulation segments of the polygon mesh comprises determining characteristic axes for each of the articulation segments, a first characteristic axis pointing in the direction of greatest spatial variation between the set of vertices in the articulation segment, a second characteristic axis pointing in the direction of greatest spatial variation between the set of vertices in the articulation segment, subject to the constraint that the second characteristic axis is perpendicular to the first characteristic axis, and a third characteristic axis being perpendicular to both the first characteristic axis and the second characteristic axis.
2. The method of claim 1 , further comprising creating the plurality of polygon meshes by scanning the physical subject in the plurality of training poses.
3. The method of claim 1 , wherein the articulation segments of the polygon mesh consist essentially of vertices which change position or orientation substantially together as a group between different poses amongst the plurality of training poses.
4. The method of claim 1 , further comprising:
identifying one of the polygon meshes as a base model; and
performing an alignment operation to align the remaining polygon meshes with the base model.
5. The method of claim 4 , wherein the alignment operation comprises an iterative closest point algorithm.
6. The method of claim 1 , wherein determining an indicator of position or orientation for each of the articulation segments of the polygon mesh comprises determining a local coordinate system for each of the articulation segments.
7. The method of claim 6 , wherein determining the local coordinate system for each of the articulation segments comprises determining a center point of the set of vertices in the articulation segment.
8. The method of claim 7 , wherein the center point comprises a centroid point.
9. The method of claim 6 , further comprising comparing the local coordinate system for each of the articulation segments with a reference coordinate system.
10. The method of claim 1 , wherein the characteristic axes comprise orthogonal principal components of the set of vertices in the articulation segment.
11. The method of claim 10 , further comprising determining the orthogonal principal components using eigen-decomposition or Singular Value Decomposition.
12. The method of claim 1 , further comprising determining a rotation matrix for each of the articulation segments which aligns a set of reference axes to the characteristic axes, or vice versa.
13. The method of claim 1 , further comprising loading a file which identifies vertices of each of the articulation segments.
14. The method of claim 1 , further comprising identifying vertices of each of the articulation segments by:
casting a plurality of rays from one of the joints; and
identifying vertices which correspond to polygon faces which are intersected by the rays.
15. The method of claim 1 , wherein the one or more joints are part of a core skeleton for the digital character.
16. The method of claim 1 , further comprising displaying the digital character using a head-mounted, see-through augmented reality display.
17. The method of claim 16 , further comprising displaying the digital character using a plurality of stacked waveguides corresponding to a plurality of depth planes.
18. The method of claim 1 , further comprising using the pose space to train the pose space deformer.
19. The method of claim 1 , further comprising:
inputting one or more joint transformations into the pose space deformer; and
outputting from the pose space deformer a new polygon mesh for the digital character, the new polygon mesh corresponding to a new pose that is not amongst the plurality of poses used to create the pose space.