IP Library Granted Patent US 12,033,263
Granted Patent B2
US 12,033,263 · App. 17/694,564 · Granted Jul 9, 2024

Using three-dimensional scans of a physical subject to determine positions and/or orientations of skeletal joints in the rigging for a virtual character

Inventor: Sean Michael Comer (Encino, CA)
Assignee: MAGIC LEAP, INC.
G06T13/40G06T7/11G06T7/73G06T17/20G06T2207/30196
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,033,263
App. No.
17/694,564
Granted
Jul 9, 2024
Kind
B2
Abstract

Systems and methods for using three-dimensional scans of a physical subject to determine positions and/or orientations of skeletal joints in the rigging for a virtual character. At least one articulation segment of a polygon mesh for the virtual character may be determined. The articulation segment may include a subset of vertices in the polygon mesh. An indicator of the position or orientation of the articulation segment of the polygon mesh may be determined. Based on the indicator of the position or orientation of the articulation segment, the position or orientation of at least one joint for deforming the polygon mesh may be determined.

Claims (32)

1. A method comprising:

segmenting a polygon mesh of a digital character into a plurality of articulation segments by casting a plurality of rays from each of a plurality of joints of a skeleton associated with the polygon mesh and identifying vertices which correspond to polygon faces which are intersected by the rays from the respective joints;

determining an indicator of position or orientation for each of the articulation segments of the polygon mesh; and

transforming one or more of the joints of the skeleton based on a transform of the indicator of position or orientation for each of the articulation segments to move the polygon mesh from a first pose to a second pose,

wherein the identifying the vertices that correspond to the polygon faces that are intersected by the rays from the respective joints includes: assigning each of the vertices to a corresponding one of the joints based on the intersection between the respect ray cast from each of the joints and the respective vertex in order to identify the articulation segments, and

wherein the identifying the vertices that correspond to the polygon faces that are intersected by the rays from the respective joints includes creating mappings between subsets of the assigned vertices and the articulation segments.

2. The method of claim 1 , wherein the polygon mesh is representative of a physical subject in a pose.

3. The method of claim 2 , further comprising scanning the physical subject while in the pose.

4. The method of claim 2 , further comprising:

segmenting each of a plurality of polygon meshes, which are representative of the physical subject in a plurality of poses, into the plurality of articulation segments, each pose comprising a unique articulation of one or more body parts of the physical subject;

determining an indicator of position or orientation for each of the articulation segments for each of the polygon meshes; and

determining, based on the indicator of position or orientation of the each of the articulation segments, the position or orientation of the one or more joints for the plurality of poses.

5. The method of claim 4 , further comprising creating a pose space for a pose space deformer based on the position or orientation of the one or more joints for the plurality of poses.

6. The method of claim 4 , further comprising creating the plurality of polygon meshes by scanning the physical subject in the plurality of poses.

7. The method of claim 4 , wherein the articulation segments of the polygon mesh consist essentially of vertices which change position or orientation substantially together as a group between different poses amongst the plurality of poses.

8. The method of claim 4 , further comprising:

identifying one of the polygon meshes as a base model; and

performing an alignment operation to align the remaining polygon meshes with the base model, the alignment operation comprising an iterative closest point algorithm.

9. The method of claim 1 , wherein determining an indicator of position or orientation for each of the articulation segments of the polygon mesh comprises determining a local coordinate system for each of the articulation segments.

10. The method of claim 9 , wherein determining the local coordinate system for each of the articulation segments comprises determining characteristic axes for each of the articulation segments.

11. The method of claim 10 , wherein the characteristic axes comprise orthogonal principal components of the set of vertices in the articulation segment.

12. The method of claim 11 , further comprising determining the orthogonal principal components using eigen-decomposition or Singular Value Decomposition.

13. The method of claim 10 , further comprising determining a first characteristic axis for each of the articulation segments, the first characteristic axis pointing in the direction of greatest spatial variation between the set of vertices in the articulation segment.

14. The method of claim 13 , further comprising determining a second characteristic axis for each of the articulation segments, the second characteristic axis pointing in the direction of greatest spatial variation between the set of vertices in the articulation segment, subject to the constraint that the second characteristic axis is perpendicular to the first characteristic axis.

15. The method of claim 14 , further comprising determining a third characteristic axis for each of the articulation segments, the third characteristic axis being perpendicular to both the first characteristic axis and the second characteristic axis.

16. The method of claim 10 , further comprising determining a rotation matrix for each of the articulation segments which aligns a set of reference axes to the characteristic axes, or vice versa.

17. The method of claim 9 , wherein determining the local coordinate system for each of the articulation segments comprises determining a center point of the set of vertices in the articulation segment, and wherein the center point comprises a centroid point.

18. The method of claim 9 , further comprising comparing the local coordinate system for each of the articulation segments with a reference coordinate system.

19. The method of claim 1 , further comprising loading a file which identifies vertices of each of the articulation segments.

20. The method of claim 1 , wherein the one or more joints are part of a core skeleton for the digital character.

21. The method of claim 1 , further comprising displaying the digital character using a head-mounted, see-through augmented reality display.

22. The method of claim 21 , further comprising displaying the digital character using a plurality of stacked waveguides corresponding to a plurality of depth planes.

Assignments (4)
SECURITY INTEREST Recorded Oct 28, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073387/0487 →
SECURITY INTEREST Recorded Oct 24, 2025
From: MAGIC LEAP, INC.; MENTOR ACQUISITION ONE, LLC; MOLECULAR IMPRINTS, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 073255/0581 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 7, 2023
From: COMER, SEAN MICHAEL
To: MAGIC LEAP, INC.
Reel/Frame 062900/0265 →
SECURITY INTEREST Recorded May 24, 2022
From: MOLECULAR IMPRINTS, INC.; MENTOR ACQUISITION ONE, LLC; MAGIC LEAP, INC.
To: CITIBANK, N.A., AS COLLATERAL AGENT
Reel/Frame 060338/0665 →