IP Library Granted Patent US 11,798,232
Granted Patent B2
US 11,798,232 · App. 17/681,530 · Granted Oct 24, 2023

Method and system for generating a new anatomy

Inventors: Jernej Barbic (Santa Monica, CA); Essex Edwards (Vancouver, CA); James Jacobs (Vancouver, CA); Crawford Doran (Vancouver, CA)
Assignee: Ziva Dynamics Inc.
G06T17/205G06T19/20G06T2219/2008
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,798,232
App. No.
17/681,530
Granted
Oct 24, 2023
Kind
B2
Abstract

Methods, systems, and techniques for generating a new anatomy use a processor to obtain a skin mesh of the new anatomy that is in correspondence with a skin mesh of a template anatomy; after obtaining the skin mesh, transfer a fascia mesh of the template anatomy to the new anatomy; and after the fascia mesh is transferred, generate a fat displacement field defining fat of the new anatomy. The displacement field includes multi-dimensional displacement vectors, and each of the displacement vectors relates a vertex of the skin mesh of the new anatomy to a corresponding vertex of the fascia mesh of the new anatomy. A new anatomy may also be generated by interpolating from anatomies in a database.

Claims (74)

1. A non-transitory computer-readable storage medium for generating a new anatomy, the medium comprising:

obtaining a skin mesh of the new anatomy that is in correspondence with a skin mesh of a template anatomy;

transferring a fascia mesh of the template anatomy to the new anatomy;

generating a fat displacement field defining fat of the new anatomy, the displacement field comprising multi-dimensional displacement vectors, each of the displacement vectors relating a vertex of the skin mesh of the new anatomy to a corresponding vertex of the fascia mesh of the new anatomy;

parameterizing the new anatomy relative to additional anatomies, a skin mesh and a fascia mesh of each of the additional anatomies in correspondence with the skin mesh and the fascia mesh of the new anatomy; and

adding the new anatomy to a database based on a comparison of one or more parameters of the new anatomy with one or more parameters of the additional anatomies.

2. The non-transitory computer-readable storage medium of claim 1 , wherein the parameterizing of the new anatomy comprises:

generating external parameters parameterizing an exterior of each of the new anatomy and the additional anatomies by performing principal components analysis on vertex positions of the skin meshes of the new anatomy and the additional anatomies to determine body eigenvectors for each of the new anatomy and the additional anatomies; and

adding the external parameters to the database.

3. The non-transitory computer-readable storage medium of claim 1 , wherein the adding the new anatomy to the database comprises:

for each of the additional anatomies, determining an exterior difference between an external parameter of the new anatomy and an external parameter of the additional anatomy and comparing the exterior difference to an external parameter difference threshold; and

adding the new anatomy to the database based on the external parameter difference threshold being satisfied for one or more of the additional anatomies.

4. The non-transitory computer-readable storage medium of claim 1 , wherein each of the additional anatomies further comprises a fat displacement field defining fat of the additional anatomy, wherein the displacement field comprises multi-dimensional displacement vectors and each of the displacement vectors relates a vertex of the skin mesh of the additional anatomy to a corresponding vertex of the fascia mesh of the additional anatomy, and wherein the parameterizing of the new anatomy comprises:

generating external parameters by parameterizing an exterior of each of the new anatomy and the additional anatomies by performing principal components analysis on vertex positions of the skin meshes of the new anatomy and the additional anatomies to determine body eigenvectors for each of the new and additional anatomies;

generating internal parameters by parameterizing an interior of each of the new anatomy and the additional anatomies by performing principal components analysis on the displacement fields of the new anatomy and the additional anatomies to determine fat eigenvectors for parameterizing the fat of each of the new anatomy and the additional anatomies; and

adding the internal parameters and the external parameters to the database.

5. The non-transitory computer-readable storage medium of claim 1 , wherein the adding the new anatomy to the database comprises:

for each of the additional anatomies, determining an exterior difference and an interior difference between an external parameter and internal parameter of the new anatomy and an external parameter and internal parameter, respectively, of the additional anatomy and comparing the exterior difference and the interior difference to a combined parameter threshold; and

adding the new anatomy to the database based on the combined parameter threshold being satisfied for one or more of the additional anatomies.

6. The non-transitory computer-readable storage medium of claim 1 , further comprising:

accessing the database storing additional anatomies; and

interpolating the new anatomy from the additional anatomies.

7. The non-transitory computer-readable storage medium of claim 6 , wherein the interpolating of the new anatomy comprises:

interpolating, using an external parameter characterizing an exterior of the new anatomy, vertices of a skin mesh of the new anatomy;

interpolating, from the database and an internal parameter characterizing fat of the new anatomy, a fat displacement field of the new anatomy comprising multidimensional displacement vectors, wherein each of the displacement vectors relates a vertex of the skin mesh to a corresponding vertex of a fascia mesh of the new anatomy; and

determining the fascia mesh of the anatomy by subtracting the fat displacement field from the vertices of the skin mesh.

8. The non-transitory computer-readable storage medium of claim 7 , further comprising:

interpolating the vertices of the skin mesh using an external starting vector, the external parameter, and body eigenvectors parameterizing exteriors of the additional anatomies; and

interpolating the fat displacement field using an internal starting vector, the internal parameter, and fat eigenvectors parameterizing fat of the additional anatomies.

9. The non-transitory computer-readable storage medium of claim 7 , further comprising, after the fascia mesh of the new anatomy is determined, transferring muscle and bone meshes from a template anatomy to the new anatomy.

10. The non-transitory computer-readable storage medium of claim 6 , wherein the interpolating of the new anatomy comprises:

obtaining a skin mesh of the new anatomy that is in correspondence with a skin mesh of a template anatomy;

determining an external parameter characterizing an exterior of the new anatomy using vertices of the skin mesh of the new anatomy and body eigenvectors determined using vertices of skin meshes of the additional anatomies;

determining an internal parameter characterizing fat of the new anatomy by using regression based on the external parameter;

interpolating, from the database and using the external parameter, vertices of an interpolated skin mesh of the new anatomy;

interpolating, from the database and using the internal parameter, a fat displacement field of the new anatomy comprising multi-dimensional displacement vectors wherein each of the displacement vectors relates a vertex of the skin mesh to a corresponding vertex of a fascia mesh of the new anatomy;

determining an interpolated fascia mesh of the new anatomy by subtracting the fat displacement field from the vertices of the interpolated skin mesh; and

determining the fascia mesh of the new anatomy by transferring the interpolated fascia mesh to the new anatomy.

11. The non-transitory computer-readable storage medium of claim 10 , further comprising, after the fascia mesh of the new anatomy is determined, transferring muscle and bone meshes from a template anatomy to the new anatomy.

12. The non-transitory computer-readable storage medium of claim 6 , wherein the interpolating of the new anatomy comprises:

obtaining a skin mesh of the new anatomy that is in correspondence with a skin mesh of a template anatomy;

determining an external parameter characterizing an exterior of the new anatomy using vertices of the skin mesh of the new anatomy and a body eigenvector determined using vertices of skin meshes of additional anatomies in a database;

determining k nearest neighbors to the external parameter in the database and weights for the nearest neighbors;

determining a weighted external parameter and a weighted internal parameter from the k nearest neighbors and the weights;

interpolating, from the database and using the weighted external parameter, vertices of an interpolated skin mesh of the new anatomy;

interpolating, from the database and using the weighted internal parameter, a fat displacement field of the new anatomy comprising multi-dimensional displacement vectors, wherein each of the displacement vectors relates a vertex of the skin mesh to a corresponding vertex of a fascia mesh of the new anatomy;

determining an interpolated fascia mesh of the new anatomy by subtracting the fat displacement field from the vertices of the interpolated skin mesh; and

determining the fascia mesh of the new anatomy by transferring the interpolated fascia mesh to the new anatomy.

13. The non-transitory computer-readable storage medium of claim 12 , further comprising, after the fascia mesh of the new anatomy is determined, transferring muscle and bone meshes from a template anatomy to the new anatomy.

14. The non-transitory computer-readable storage medium of claim 6 , wherein the interpolating of the new anatomy comprises:

obtaining a skin mesh of the new anatomy that is in correspondence with a skin mesh of a template anatomy;

determining k nearest neighbors to the skin mesh of the new anatomy and weights for the nearest neighbors; and

determining a fascia of the new anatomy by weight summing fascia meshes of the nearest neighbors.

15. The non-transitory computer-readable storage medium of claim 14 , further comprising, after the fascia mesh of the new anatomy is determined, transferring muscle and bone meshes from a template anatomy to the new anatomy.

16. The non-transitory computer-readable storage medium of claim 14 , further comprising determining at least one of bone meshes of the new anatomy by weight summing bone meshes of the nearest neighbors and muscle meshes of the new anatomy by weight summing muscle meshes of the nearest neighbors.

17. A system comprising:

one or more computer processors;

one or more computer memories;

a set of instructions incorporated into the one or more computer memories, the set of instructions configuring the one or more computer processors to perform operations, the operations comprising:

transferring a fascia mesh of the template anatomy to the new anatomy;

generating a fat displacement field defining fat of the new anatomy, the displacement field comprising multi-dimensional displacement vectors, each of the displacement vectors relating a vertex of the skin mesh of the new anatomy to a corresponding vertex of the fascia mesh of the new anatomy;

parameterizing the new anatomy relative to additional anatomies, a skin mesh and a fascia mesh of each of the additional anatomies in correspondence with the skin mesh and the fascia mesh of the new anatomy; and

adding the new anatomy to a database based on a comparison of one or more parameters of the new anatomy with one or more parameters of the additional anatomies.

18. The system of claim 17 , the operations further comprising:

accessing the database storing additional anatomies; and

interpolating the new anatomy from the additional anatomies.

19. A method for storing a set of instructions that, when executed by one or more computer processors, cause the one or more computer processors to perform operations, the operations comprising:

transferring a fascia mesh of the template anatomy to the new anatomy;

generating a fat displacement field defining fat of the new anatomy, the displacement field comprising multi-dimensional displacement vectors, each of the displacement vectors relating a vertex of the skin mesh of the new anatomy to a corresponding vertex of the fascia mesh of the new anatomy;

parameterizing the new anatomy relative to additional anatomies, a skin mesh and a fascia mesh of each of the additional anatomies in correspondence with the skin mesh and the fascia mesh of the new anatomy; and

adding the new anatomy to a database based on a comparison of one or more parameters of the new anatomy with one or more parameters of the additional anatomies.

20. The method of claim 19 , the operations further comprising:

accessing the database storing additional anatomies; and

interpolating the new anatomy from the additional anatomies.

Assignments (3)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 24, 2025
From: UNITY TECHNOLOGIES CANADA COMPANY
To: UNITY TECHNOLOGIES SF
Reel/Frame 072355/0596 →
MERGER AND CHANGE OF NAME Recorded Jul 6, 2022
From: ZIVA DYNAMICS INC.; UNITY TECHNOLOGIES CANADA COMPANY
To: UNITY TECHNOLOGIES CANADA COMPANY
Reel/Frame 060614/0250 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 28, 2022
From: BARBIC, JERNEJ; EDWARDS, ESSEX; JACOBS, JAMES; DORAN, CRAWFORD
To: ZIVA DYNAMICS, INC.
Reel/Frame 059124/0584 →
Continuity (3)
Continuation 16635979
Provisional Application 62540539 · Aug 2, 2017
Related Publication 20220180607A1 · Jun 9, 2022