IP Library › Granted Patent US 10,854,005
Granted Patent B2
US 10,854,005 · App. 16/122,424 · Granted Dec 1, 2020

Visualization of ultrasound images in physical space

Inventors: Sean A. Lisse (Woodbridge, CT); Fabian Max Laage-Gaupp (New Haven, CT); Andreas Erben (New Haven, CT)
G06T19/006A61B8/08A61B8/44G06T7/0012G06T7/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,854,005
App. No.
16/122,424
Filed
Sep 5, 2018
Granted
Dec 1, 2020
Kind
B2
Art Unit
2662
USPC
382/128
Abstract

Systems and methods of real-time augmented visualization of anatomical features are disclosed herein. The systems and methods can use concurrently collected and/or preexisting data regarding an anatomical structure to present an image of the anatomical structure to an operator in virtual space as overlayed in real space. The systems and methods can include acquiring image data from an anatomical structure of a subject. Visual image data of the subject can then be received, including non-rigid deformation of the subject. The device pose of the ultrasound device in real space can be determined using device pose data. Image pose of the image data can then be determined in virtual space. Then, a perspective pose of the image of at least a portion of the anatomical structure in virtual space can be mapped to the subject in real space.

Claims (37)

1. An anatomical presentation system for real-time augmented visualization of anatomical features, comprising:

one or more processors; and

a memory communicably coupled to the one or more processors and storing:

an acquisition module comprising instructions that when executed by the one or more processors cause the one or more processors to acquire image data, using an ultrasound device, from an anatomical structure of a subject and to receive visual image data of the subject, the image data including deformation data for the anatomical structure, the deformation data being data collected about the mobility of the anatomical structure in response to one or more forces;

a coordination module comprising instructions that when executed by the one or more processors cause the one or more processors to determine a device pose of the ultrasound device in real space, and to determine image pose data of the image data and the visual image data in virtual space, with relation to the device pose of the ultrasound device, the subject, and an operator in real space; and

a visualization module comprising instructions that when executed by the one or more processors cause the one or more processors to map a perspective pose of an anatomical structure image in virtual space to the subject in real space, the anatomical structure image being a visual representation of the anatomical structure constructed from at least the image data, the visual image data and the image pose data, the anatomical structure image incorporating one or more visualizations of the non-rigid deformation of the anatomical structure, in response to the one or more forces, as derived from the deformation data.

2. The anatomical presentation system of claim 1 , wherein the acquisition module further comprises instructions to collect subject deformation data for the subject, and to predict one or more deformations of the anatomical structure using the subject deformation data.

3. The anatomical presentation system of claim 2 , wherein the acquisition module further comprises, instructions to predict one or more deformations of a secondary structure in the subject, based on the predicted deformations of the anatomical structure.

4. The anatomical presentation system of claim 1 , wherein the image data comprises a first image data collected at a first period of time about the anatomical structure and real-time image data concurrently collected with the visual image data about the anatomical structure, the first, image data and the real-time image data being merged to produce the anatomical structure image.

5. The anatomical presentation system of claim 1 , wherein the image data comprises an ultrasound image data and secondary image data collected using, a secondary imaging technique.

6. The anatomical presentation system of claim 5 , wherein the coordination module further comprises instructions to produce a hybrid data set from the ultrasound image data and the secondary image data, the hybrid data set being used to produce the anatomical structure image.

7. The anatomical presentation system of claim 1 , wherein the visualization module further comprises instructions to render obstructing objects transparent to the operator.

8. A non-transitory computer-readable medium for real-time augmented visualization of anatomical features and storing instructions that when executed by one or more processors cause the one or more processors to:

acquire image data, using an ultrasound device, from an anatomical structure of a subject;

receive visual image data of the subject, the image data including deformation data for the anatomical structure, the deformation data being data collected about the mobility of the anatomical structure in response to one or more forces;

determine device pose of the ultrasound device in real space;

determine image pose data of the image data and the visual image data in virtual space, with relation to the device pose of the ultrasound device, the subject, and an operator in real space; and

map a perspective pose of an anatomical structure image in virtual space to the subject in real space, the anatomical structure image being a visual representation of the anatomical structure constructed from at least the image data, the visual image data, and the image pose data, the anatomical structure image incorporating one or more visualizations of the non-rigid deformation of the anatomical structure, in response to the one or more forces, as derived from the deformation data.

9. The non-transitory computer-readable medium of claim 8 , further comprising instructions to collect subject deformation data for the subject, and predict one or more deformations of the anatomical structure using the subject deformation data.

10. The non-transitory computer-readable medium of claim 9 , further comprising instructions to predict one or more deformations of a secondary structure in the subject, based on the predicted deformations of the anatomical structure.

11. The non-transitory computer-readable medium of claim 8 , wherein the image data comprises a first image data collected at a first period of time about the anatomical structure and second image data concurrently collected with the visual image data about the anatomical structure, the first image data and the second image data being merged to produce the anatomical structure image.

12. The non-transitory computer-readable medium of claim 8 , wherein the image data comprises an ultrasound image data and secondary image data collected using a secondary imaging technique.

13. The non-transitory computer-readable medium of claim 12 , further comprising instructions to produce a hybrid data set from the ultrasound image data and the secondary image data, the hybrid, data set being used to produce the anatomical structure image.

14. The non-transitory computer-readable medium of claim 8 , further comprising instructions to render obstructing objects transparent to the operator.

15. A method for real-time augmented visualization of anatomical features, comprising:

acquiring image data, using an ultrasound device, from an anatomical structure of a subject;

receiving visual image data of the subject, the image data including deformation data for the anatomical structure, the deformation data being data collected about the mobility of the anatomical structure in response to one or more forces;

determining device pose of the ultrasound device in real space;

determining image pose data of the image data and the visual image data in virtual space, with relation to the device pose of the ultrasound device, the subject, and an operator in real space; and

mapping a perspective pose of an anatomical structure image in virtual space to the subject in real space, the anatomical structure image being a visual representation of the anatomical structure constructed from at least the image data, the visual image data, and the image pose data, the anatomical structure image incorporating one or more visualizations of the non-rigid deformation of the anatomical structure, in response to the one or more forces, as derived from the deformation data.

16. The method of claim 15 , further comprising:

collectins subject deformation data for the subject; and

predicting one or more deformations of the anatomical structure using the subject deformation data.

17. The method of claim 15 , further comprising predicting, one or more deformations of a secondary structure in the subject, based on the predicted deformations of the anatomical structure.

18. The method of claim 15 , wherein the image data comprises a first image data collected at a first period of time about the anatomical structure and second image data concurrently collected with the visual image data about the anatomical structure, the first image data and the second image data being merged to produce the anatomical structure image.

19. The method of claim 15 , wherein the image data comprises an ultrasound image data and secondary image data collected using a secondary imaging technique, and further comprising producing a hybrid data set from the ultrasound image data and the secondary image data, the hybrid data set being used to produce the anatomical structure image.

20. The method of claim 15 , further comprising rendering obstructing objects transparent to the operator.

Continuity (1)
Related Publication 20200074737A1 · Mar 5, 2020