Granted Patent B2

US 12,440,290 · App. 18/153,193 · Granted Oct 14, 2025

Inventors: Mahdi Azizian (San Jose, CA); Simon P. DiMaio (San Carlos, CA)

Assignee: INTUITIVE SURGICAL OPERATIONS, INC.

A61B34/30A61B1/00009A61B1/000094A61B1/00194A61B34/20A61B34/35A61B2034/105A61B2034/2051A61B2034/301A61B2090/3762

Abstract

Claims (30)

1. A method comprising:

obtaining an endoscopic image dataset of a patient anatomy from an endoscopic imaging system, wherein the endoscopic image dataset comprises a plurality of two-dimensional endoscopic images;

retrieving an anatomic model dataset of the patient anatomy obtained by an external anatomic imaging system;

mapping the plurality of two-dimensional endoscopic images of the endoscopic image dataset onto a three-dimensional volume defined by the anatomic model dataset;

constructing, during a medical procedure, a first three-dimensional vantage point image using the mapped endoscopic image dataset;

displaying the first three-dimensional vantage point image such that the first three-dimensional vantage point image is presented from a first vantage point of a distal end of the endoscopic imaging system, the first vantage point corresponding to a first location of the distal end of the endoscopic imaging system;

generating a first three-dimensional anatomic model image from the anatomic model dataset;

displaying, during the medical procedure, the first three-dimensional anatomic model image, with the first three-dimensional vantage point image, by overlaying the first three-dimensional vantage point image on the first three-dimensional anatomic model image;

receiving user instructions to move the distal end of the endoscopic imaging system from the first location to a second location;

constructing, during the medical procedure, a second three-dimensional vantage point image using the mapped endoscopic image dataset; and

displaying the second three-dimensional vantage point image such that the second three-dimensional vantage point image is presented from a second vantage point of the distal end of the endoscopic imaging system, the second vantage point corresponding to the second location of the distal end of the endoscopic imaging system, the second vantage point being different from the first vantage point.

2. The method of claim 1 wherein the first and second vantage points are inside the patient anatomy.

3. The method of claim 1 wherein mapping the endoscopic image dataset to the anatomic model dataset includes mapping a two dimensional point cloud from the endoscopic image dataset into a three dimensional point cloud based on the anatomic model dataset.

4. The method of claim 3 wherein constructing the first three-dimensional vantage point image includes constructing the first three-dimensional vantage point image from the three dimensional point cloud.

5. The method of claim 1 further comprising:

displaying a second three-dimensional anatomic model image, generated from the anatomic model dataset, with the second three-dimensional vantage point image.

6. The method of claim 1 wherein displaying the first three-dimensional anatomic model image with the first three-dimensional vantage point image includes depth matching the endoscopic image dataset to the anatomic model dataset.

7. The method of claim 1 wherein displaying the first three-dimensional anatomic model image with the first three-dimensional vantage point image includes scaling the first three-dimensional anatomic model image to the first three-dimensional vantage point image.

8. The method of claim 1 wherein the endoscopic image dataset includes a first two dimensional image dataset and the anatomic model dataset includes a three-dimensional anatomic model dataset, and wherein the mapping includes mapping the first two dimensional image dataset to the three-dimensional anatomic model dataset.

9. The method of claim 8 wherein the endoscopic image dataset includes a second two dimensional image dataset such that the first and second two dimensional image datasets form a stereoscopic image dataset, and wherein the mapping includes mapping the second two dimensional image dataset to the three-dimensional anatomic model dataset.

10. The method of claim 1 wherein the external anatomic imaging system is a computed tomography imaging system, a magnetic resonance imaging system, or an ultrasound imaging system.

11. The method of claim 1 further comprising:

tracking a medical instrument positioned within the patient anatomy; and

displaying a simulated image of the tracked medical instrument extending from the patient anatomy with the displayed second three-dimensional vantage point image.

12. The method of claim 11 , wherein the simulated image of the tracked medical instrument is generated based on a position of the medical instrument generated from shape sensor information.

13. The method of claim 11 , wherein the simulated image of the tracked medical instrument is generated based on a position of the medical instrument generated from kinematic chain information.

14. The method of claim 11 , wherein the simulated image of the tracked medical instrument is generated based on a position of the medical instrument generated from a camera using vision-based tracking.

15. The method of claim 1 , wherein the endoscopic image dataset comprises live image data and recorded image data.

16. The method of claim 1 , wherein the first three-dimensional vantage point image includes a visual property that distinguishes the first three-dimensional vantage point image from the first three-dimensional anatomic model image.

17. The method of claim 1 , wherein the first three-dimensional vantage point image is at least partially transparent so that a portion of the first three-dimensional anatomic model image is visible through the first three-dimensional vantage point image.

Assignments (1)

Continuity (3)

References Cited (39)