Systems and methods for automated rigging and generating animations based on real musculoskeletal movements

1 . A method comprising:

receiving a plurality of scans of a target subject;

generating a first plurality of primitives that form a three-dimensional (3D) model of the target subject based on a first scan of the plurality of scans;

defining a musculoskeletal framework comprising a second plurality of primitives that represent virtual muscles and virtual bones detected from other scans of the plurality of scans;

rigging the 3D model for animation based on an alignment of the second plurality of primitives of the musculoskeletal framework and the first plurality of primitives of the 3D model; and

animating the 3D model based on simulated contractions of one or more of the virtual muscles, wherein animating the 3D model comprises:

determining an association between primitives from the second plurality of primitives representing the one or more virtual muscles and aligned primitives of the first plurality of primitives; and

adjusting the aligned primitives according to a movement created from the simulated contractions of the one or more virtual muscles.

2 . The method of claim 1 further comprising:

generating the virtual bones of the musculoskeletal framework based on scanned bones of the target subject captured in a second scan of the plurality of scans.

3 . The method of claim 2 , wherein generating the virtual bones comprises:

converting a scanned bone in the second scan to a subset of primitives in the second plurality of primitives that represent the scanned bone in the musculoskeletal framework at a size and position that matches a size and position of the scanned bone in the second scan.

4 . The method of claim 2 further comprising:

generating the virtual muscles of the musculoskeletal framework based on imaged muscles of the target subject captured in a third scan of the plurality of scans.

5 . The method of claim 4 further comprising:

connecting the virtual muscles to different points about the virtual bones based on an alignment of the scanned bones from the second scan with the imaged muscles from the third scan.

6 . The method of claim 1 , wherein defining the musculoskeletal framework comprises

converting x-ray data to a first set of points or meshes of a 3D format that correspond to a first set of primitives from the second plurality of primitives; and

converting ultrasound data to a second of points or meshes of the 3D format that correspond to a second set of primitives from the second plurality of primitives.

7 . The method of claim 1 further comprising:

mapping the second plurality of primitives representing the virtual muscles and the virtual bones of the musculoskeletal framework to different sets of the first plurality of primitives of the 3D model based on alignment of the plurality of scans and a proximity between the different sets of primitives and primitives representing different virtual muscles of the musculoskeletal framework.

8 . The method of claim 1 further comprising:

receiving training data showing movements of the target subject or a related subject;

mapping the virtual muscles and the virtual bones to different parts of the target subject or the related subject in the training data;

determining a simulated contraction of a particular virtual muscle that recreates a motion of a mapped part of the target subject or the related subject across the training data; and

associating the simulated contraction to the particular virtual muscle in an animation model of the musculoskeletal framework.

9 . The method of claim 8 , wherein determining the simulated contraction comprises:

animating the particular virtual muscle with a plurality of functions; and

selecting a particular function from the plurality of functions that animates the particular virtual muscle with the motion of the mapped part.

10 . The method of claim 8 , wherein animating the 3D model further comprises:

defining a movement of the particular virtual muscle that animates the primitives of the second plurality of primitives from a starting position to an end position;

adjusting a shape or form of the particular virtual muscle across the movement of the particular virtual muscle according to the simulated contraction; and

wherein adjusting the aligned primitives comprises repositioning the aligned primitives to match the shape or form of the primitives of the second plurality of primitives across the movement of the particular virtual muscle.

11 . The method of claim 1 , wherein the first plurality of primitives correspond to points of a point cloud or meshes of a mesh model.

12 . The method of claim 1 , wherein adjusting the aligned primitives comprises:

applying the movement created from the simulated contractions of the one or more virtual muscles to the aligned primitives.

13 . The method of claim 1 , wherein adjusting the aligned primitives comprises:

determining that the aligned primitives represent an outer layer for a particular part of the 3D model and that the primitives from the second plurality of primitives represent an inner structure for the particular part of the 3D model; and

repositioning the aligned primitives representing the outer layer to match a changing shape or form of the inner structure defined from the simulated contractions of the one or more virtual muscles.

14 . The method of claim 1 , wherein animating the 3D model further comprises:

rendering the first plurality of primitives with a positioning of the aligned primitives changing according to the movement created from the simulated contractions of the one or more virtual muscles and without a visualization of the one or more virtual muscles or the second plurality of primitives.

15 . The method of claim 1 further comprising:

presenting an animation comprising a changing visualization of the first plurality of primitives with a positioning of the aligned primitives changing according to the movement created from the simulated contractions of the one or more virtual muscles, wherein the one or more virtual muscles and the second plurality of primitives are not rendered as part of the animation or the changing visualization.

16 . A three-dimensional (3D) animation system comprising:

one or more hardware processors configured to:

receive a plurality of scans of a target subject;

generate a first plurality of primitives that form a 3D model of the target subject based on a first scan of the plurality of scans;

define a musculoskeletal framework comprising a second plurality of primitives that represent virtual muscles and virtual bones detected from other scans of the plurality of scans;

rig the 3D model for animation based on an alignment of the second plurality of primitives of the musculoskeletal framework and the first plurality of primitives of the 3D model; and

animate the 3D model based on simulated contractions of one or more of the virtual muscles, wherein animating the 3D model comprises:

determining an association between primitives from the second plurality of primitives representing the one or more virtual muscles and aligned primitives of the first plurality of primitives; and

adjusting the aligned primitives according to a movement created from the simulated contractions of the one or more virtual muscles.

17 . The system of claim 16 , wherein the one or more hardware processors are further configured to:

generate the virtual bones of the musculoskeletal framework based on scanned bones of the target subject captured in a second scan of the plurality of scans.

18 . The system of claim 17 , wherein generating the virtual bones comprises:

converting a scanned bone in the second scan to a subset of primitives in the second plurality of primitives that represent the scanned bone in the musculoskeletal framework at a size and position that matches a size and position of the scanned bone in the second scan.

19 . The system of claim 17 , wherein the one or more hardware processors are further configured to:

generate the virtual muscles of the musculoskeletal framework based on imaged muscles of the target subject captured in a third scan of the plurality of scans.

20 . A non-transitory computer-readable medium storing instructions that, when executed by one or more processors of a three-dimensional (3D) animation system, cause the 3D animation system to perform operations comprising:

receiving a plurality of scans of a target subject;

generating a first plurality of primitives that form a 3D model of the target subject based on a first scan of the plurality of scans;

defining a musculoskeletal framework comprising a second plurality of primitives that represent virtual muscles and virtual bones detected from other scans of the plurality of scans;

rigging the 3D model for animation based on an alignment of the second plurality of primitives of the musculoskeletal framework and the first plurality of primitives of the 3D model; and

animating the 3D model based on simulated contractions of one or more of the virtual muscles, wherein animating the 3D model comprises:

determining an association between primitives from the second plurality of primitives representing the one or more virtual muscles and aligned primitives of the first plurality of primitives; and

adjusting the aligned primitives according to a movement created from the simulated contractions of the one or more virtual muscles.