IP Library › Granted Patent US 12,226,202
Granted Patent B2
US 12,226,202 · App. 18/405,859 · Granted Feb 18, 2025

Systems and methods for registration of multiple vision systems

Inventors: Tao Zhao (Sunnyvale, CA); Prashant Chopra (Foster City, CA)
Assignee: INTUITIVE SURGICAL OPERATIONS, INC.
A61B5/066A61B1/000094A61B1/009A61B34/20G06T7/33A61B5/062A61B2034/2065G06T2207/10068G06T2207/30021
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,226,202
App. No.
18/405,859
Granted
Feb 18, 2025
Kind
B2
Abstract

A system comprises a processor and a memory storing computer-readable instructions that, when executed by the processor, cause the system to: define a subregion of a model of an anatomic region, the subregion including a plurality of virtual tissue structures and corresponding to an area surrounding a tip of an image capture probe located within the anatomic region; compare a tissue structure in an image received from the image capture probe to at least a portion of virtual tissue structures of the plurality of virtual tissue structures; based on the comparison, determine a respective similarity measure between the image and each virtual tissue structure; identify a closest matched virtual tissue structure based on the determined similarity measures; and based on identifying the closest matched virtual tissue structure, register the image to the model to determine a virtual position for the image capture probe with respect to the model.

Claims (45)

1. A system comprising:

a processor; and

a memory storing computer-readable instructions that, when executed by the processor, cause the system to:

define a subregion of a model of an anatomic region of a patient anatomy, the subregion corresponding to an area surrounding a tip of an image capture probe located within the anatomic region, the subregion including a plurality of virtual tissue structures;

compare a tissue structure in an image received from the image capture probe to at least a portion of virtual tissue structures of the plurality of virtual tissue structures;

based on the comparison, determine a respective similarity measure between the image received from the image capture probe and each virtual tissue structure of the portion of virtual tissue structures;

identify a closest matched virtual tissue structure based on the determined similarity measures; and

based on identifying the closest matched virtual tissue structure, register the image received from the image capture probe to the model of the anatomic region to determine a virtual position for the image capture probe with respect to the model.

2. The system of claim 1 , wherein identifying the closest matched virtual tissue structure includes identifying a greatest similarity measure of the determined similarity measures.

3. The system of claim 1 , wherein comparing the tissue structure in the image received from the image capture probe to the portion of virtual tissue structures includes:

identifying structural aspects of the tissue structure in the image; and

comparing the identified structural aspects to structural aspects of each virtual tissue structure of the portion of virtual tissue structures.

4. The system of claim 1 , wherein the image capture probe is positioned within an elongate flexible device, and wherein the computer-readable instructions, when executed by the processor, further cause the system to:

guide the elongate flexible device through the anatomic region based on the registration between the image received from the image capture probe and the model of the anatomic region.

5. The system of claim 1 , wherein the portion of virtual tissue structures of the plurality of virtual tissue structures is situated distally from the tip of the image capture probe.

6. The system of claim 1 , wherein the tissue structure in the image received from the image capture probe is a bifurcation of the patient anatomy, and wherein each virtual tissue structure of the portion of virtual tissue structures is a virtual bifurcation.

7. The system of claim 6 , wherein comparing the tissue structure in the image received from the image capture probe to the portion of virtual tissue structures of the plurality of virtual tissue structures comprises comparing the bifurcation of the patient anatomy to each virtual bifurcation.

8. The system of claim 1 , wherein the computer-readable instructions, when executed by the processor, further cause the system to:

determine a position of the tip of the image capture probe based on information received from a sensor system.

9. The system of claim 1 , wherein the computer-readable instructions, when executed by the processor, further cause the system to:

generate the model of the anatomic region, wherein the model is three-dimensional.

10. The system of claim 1 , wherein the computer-readable instructions, when executed by the processor, further cause the system to:

generate a registered virtual image from the model based registering the image received from the image capture probe to the model of the anatomic region.

11. The system of claim 10 , wherein the computer-readable instructions, when executed by the processor, further cause the system to:

display the registered virtual image.

12. The system of claim 11 , wherein the image capture probe comprises a camera.

13. The system of claim 10 , wherein the computer-readable instructions, when executed by the processor, further cause the system to:

display information extracted from the registered virtual image overlaid on the image received from the image capture probe.

14. A method comprising:

defining a subregion of a model of an anatomic region of a patient anatomy, the subregion corresponding to an area surrounding a tip of an image capture probe located within the anatomic region, the subregion including a plurality of virtual tissue structures;

comparing a tissue structure in an image received from the image capture probe to at least a portion of virtual tissue structures of the plurality of virtual tissue structures;

based on the comparison, determining a respective similarity measure between the image received from the image capture probe and each virtual tissue structure of the portion of virtual tissue structures;

identifying a closest matched virtual tissue structure based on the determined similarity measures; and

based on identifying the closest matched virtual tissue structure, registering the image received from the image capture probe to the model of the anatomic region to determine a virtual position for the image capture probe with respect to the model.

15. The method of claim 14 , wherein identifying the closest matched virtual tissue structure includes identifying a greatest similarity measure of the determined similarity measures.

16. The method of claim 14 , wherein comparing the tissue structure in the image received from the image capture probe to the portion of virtual tissue structures includes:

identifying structural aspects of the tissue structure in the image; and

comparing the identified structural aspects to structural aspects of each virtual tissue structure of the portion of virtual tissue structures.

17. The method of claim 14 , wherein the image capture probe is positioned within an elongate flexible device, and wherein the method further comprises:

guide the elongate flexible device through the anatomic region based on the registration between the image received from the image capture probe and the model of the anatomic region.

18. The method of claim 14 , further comprising:

generate a registered virtual image from the model based registering the image received from the image capture probe to the model of the anatomic region.

19. The method of claim 18 , further comprising:

display either the registered virtual image or information extracted from the registered virtual image overlaid on the image received from the image capture probe.

20. The method of claim 14 , wherein the tissue structure in the image received from the image capture probe is a bifurcation of the patient anatomy, wherein each virtual tissue structure of the portion of virtual tissue structures is a virtual bifurcation, and wherein comparing the tissue structure in the image received from the image capture probe to the portion of virtual tissue structures of the plurality of virtual tissue structures comprises comparing the bifurcation of the patient anatomy to each virtual bifurcation.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 15, 2024
From: ZHAO, TAO; CHOPRA, PRASHANT
To: INTUITIVE SURGICAL OPERATIONS, INC.
Reel/Frame 066118/0767 →
Continuity (5)
Continuation 17532039 · Nov 22, 2021
Continuation 16353778 · Mar 14, 2019
Continuation 13964704 · Aug 12, 2013
Provisional Application 61682966 · Aug 14, 2012
Related Publication 20240252057A1 · Aug 1, 2024
References Cited (57)
US 5495540A · Frankot et al. · 1996 [cited by applicant]
US 6380732B1 · Gilboa · 2002 [cited by applicant]
US 6389187B1 · Greenaway et al. · 2002 [cited by applicant]
US 7756563B2 · Higgins et al. · 2010 [cited by applicant]
US 7889905B2 · Higgins et al. · 2011 [cited by applicant]
US 7901348B2 · Soper et al. · 2011 [cited by applicant]
US 7930065B2 · Larkin et al. · 2011 [cited by applicant]
US 10278615B2 · Zhao et al. · 2019 [cited by applicant]
US 11219385B2 · Zhao et al. · 2022 [cited by applicant]
US 11896364B2 · Zhao et al. · 2024 [cited by applicant]
US 20040034300A1 · Verard et al. · 2004 [cited by applicant]
US 20040249267A1 · Gilboa, II et al. · 2004 [cited by applicant]
US 20050015005A1 · Kockro · 2005 [cited by applicant]
US 20050182295A1 · Soper · 2005 [cited by examiner]
US 20060013523A1 · Childlers et al. · 2006 [cited by applicant]
US 20060149134A1 · Soper et al. · 2006 [cited by applicant]
US 20060184016A1 · Glossop · 2006 [cited by applicant]
US 20060281971A1 · Sauer et al. · 2006 [cited by applicant]
US 20070053564A1 · Matsumoto · 2007 [cited by applicant]
US 20070179496A1 · Swoyer et al. · 2007 [cited by applicant]
US 20070293721A1 · Gilboa · 2007 [cited by applicant]
US 20080004633A1 · Arata et al. · 2008 [cited by applicant]
US 20080207997A1 · Higgins et al. · 2008 [cited by applicant]
US 20080212082A1 · Froggatt et al. · 2008 [cited by applicant]
US 20090054729A1 · Mori et al. · 2009 [cited by applicant]
US 20090161927A1 · Mori et al. · 2009 [cited by applicant]
US 20090252395A1 · Chan et al. · 2009 [cited by applicant]
US 20090268955A1 · Koolwal et al. · 2009 [cited by applicant]
US 20090292175A1 · Akimoto et al. · 2009 [cited by applicant]
US 20100041949A1 · Tolkowsky · 2010 [cited by applicant]
US 20100239147A1 · Vitanovski et al. · 2010 [cited by applicant]
US 20100239152A1 · Furst · 2010 [cited by applicant]
US 20100280365A1 · Higgins et al. · 2010 [cited by applicant]
US 20110128352A1 · Higgins et al. · 2011 [cited by applicant]
US 20110184238A1 · Higgins et al. · 2011 [cited by applicant]
US 20120069167A1 · Liu · 2012 [cited by examiner]
US 20120289777A1 · Chopra et al. · 2012 [cited by applicant]
US 20220079467A1 · Zhao et al. · 2022 [cited by applicant]
CN 1658789A · 2005 [cited by applicant]
CN 102186404A · 2011 [cited by applicant]
EP 2423877A1 · 2012 [cited by applicant]
JP 2005131043A · 2005 [cited by applicant]
JP 2008541860A · 2008 [cited by applicant]
JP 2009279251A · 2009 [cited by applicant]
JP 2012024518A · 2012 [cited by applicant]
WO WO2008095068A1 · 2008 [cited by applicant]
WO WO2010078009A1 · 2010 [cited by applicant]
WO WO2010101660A2 · 2010 [cited by applicant]
WO WO2011086431A1 · 2011 [cited by applicant]
WO WO2012035492A1 · 2012 [cited by applicant]
WO WO2012095755A1 · 2012 [cited by applicant]
Bricault L., et al., “Registration of Real and CT-derived Virtual Bronchoscopic Images to Assist Transbronchial Biopsy,” IEEE Transactions on Medical Imaging, Oct. 1998, vol. 17 (5), pp. 703-714. [cited by applicant]
Extended European Search Report for Application No. EP13829698.3, mailed on Mar. 29, 2016, 11 pages (ISRG04530/EP). [cited by applicant]
Extended European Search Report for Application No. EP19209475.3, mailed on Jun. 17, 2020, 13 pages (P00464-EP-02). [cited by applicant]
International Search Report and Written Opinion for Application No. PCT/US13/54555, mailed on Nov. 27, 2013, 14 pages (ISRG04530/PCT). [cited by applicant]
Soper T.D., et al., “In Vivo Validation of a Hybrid Tracking System for Navigation of an Ultrathin Bronchoscope within Peripheral Airways,” IEEE Transactions on Biomedical Engineering, 2010, vol. 57 (3), pp. 736-745. [cited by applicant]
Vertut, J., and Coiffet, P., “Robot Technology: Teleoperation and Robotics Evolution and Development,” English translation, Prentice-Hall, Inc., Inglewood Cliffs, NJ, USA 1986, vol. 3A, 332 pages. [cited by applicant]