IP Library Granted Patent US 10,482,599
Granted Patent B2
US 10,482,599 · App. 16/372,093 · Granted Nov 19, 2019

Navigation of tubular networks

Inventors: David S. Mintz (Mountain View, CA); Atiyeh Ghoreyshi (Richmond, CA); Prasanth Jeevan (San Mateo, CA); Yiliang Xu (Cupertino, CA); Gehua Yang (Wexford, PA); Matthew Joseph Leotta (Clifton Park, NY); Charles V. Stewart (Clifton Park, NY)
Assignee: Auris Health, Inc.
G06T7/0012A61B1/005A61B1/00009A61B1/0016A61B1/00147A61B1/00149A61B1/04A61B1/2676A61B5/061A61B6/032A61B34/20G06T7/149G06T7/248G06T7/32G06T7/73G06T15/205G06T17/00A61B34/25A61B90/30A61B2017/00809A61B2034/105A61B2034/107A61B2034/2048A61B2034/2051A61B2034/2055A61B2034/252A61B2034/301A61B2090/3614G06T2207/10068G06T2207/10081G06T2207/30061G06T2207/30196
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,482,599
App. No.
16/372,093
Granted
Nov 19, 2019
Kind
B2
Abstract

Methods and apparatuses provide improved navigation through tubular networks such as lung airways by providing improved estimation of location and orientation information of a medical instrument (e.g., an endoscope) within the tubular network. Various input data such as image data, EM data, and robot data are used by different algorithms to estimate the state of the medical instrument, and the state information is used to locate a specific site within a tubular network and/or to determine navigation information for what positions/orientations the medical instrument should travel through to arrive at the specific site. Probability distributions together with confidence values are generated corresponding to different algorithms are used to determine the medical instrument's estimated state.

Claims (58)

1. A robotic medical system for bronchoscopy, the system comprising:

a robotic arm comprising a plurality of segments, the robotic arm extending from a base;

a robotically-controllable bronchoscope removably coupled to the robotic arm, the bronchoscope configured for insertion into a bronchial network of a patient, wherein a distal end of the bronchoscope is articulable; and

a command console comprising a command module configured to allow an operator to remotely control the bronchoscope, wherein the command module is configured to allow the operator to remotely articulate the distal end bronchoscope, and wherein the command module comprises a trackball;

a display;

at least one computer-readable medium having stored thereon instructions and at least one processor in communication with the at least one computer-readable medium and configured to execute the instructions to cause a navigation module to:

display a model of the bronchial network;

access a selected target location in the bronchial network;

access a determined path from an entry point to the target location;

receive input data related to a position of the bronchoscope from a plurality of data sources;

determine the position of the bronchoscope within the bronchial network based on the input data from one or more of the plurality of data sources;

display the determined position of the bronchoscope relative to the displayed model of the bronchial network; and

display navigational guidance on the display, the navigational guidance comprising the determined path to the target location, a distance to the target location, a location of the bronchoscope relative to the model of the bronchial network, and depth data.

2. The system of claim 1 , wherein the navigation module is configured to:

display the target location on the displayed model of the bronchial network; and

display the target location relative to a position and orientation of the bronchoscope.

3. The system of claim 1 , wherein the plurality of data sources comprises at least one of:

position sensor data received from a position sensor on the bronchoscope,

image data received from an imaging device on the bronchoscope,

robot data comprising command data for instructing the system to change the orientation of the bronchoscope, and

model data.

4. The system of claim 3 , wherein the robot data comprises at least one of:

a pitch, a roll, and a yaw of the bronchoscope; and

an insertion distance of the bronchoscope into the bronchial network.

5. The system of claim 3 , wherein the model data comprises a computer model of the bronchial network generated based on computerized axial tomography (CT) scans of the tubular network.

6. The system of claim 1 , wherein the determined path comprises a path along centerlines of lumens of the model.

7. The system of claim 1 , wherein the base comprises a cart with wheels.

8. The system of claim 1 , wherein the robotic arm comprises at least one set-up joint including a brake and counter-balance to maintain a set-up position of the robotic arm.

9. The system of claim 1 , wherein the navigation module is configured to display a computer model of the bronchoscope relative to the model of the bronchial network.

10. The system of claim 1 , wherein the display comprises two or more display modules.

11. The system of claim 1 , wherein the bronchoscope includes an imaging device, and wherein a biopsy tool or surgical instrument can be inserted into a working channel of the bronchoscope without removing the imaging device.

12. A method for robotically-assisted bronchoscopy, the method comprising:

displaying a model of a bronchial network;

accessing a selected target location in the bronchial network;

accessing a determined path from an entry point of the bronchial network to the target location;

inserting a robotically-controllable bronchoscope into a bronchial network of a patient with a robotic arm comprising a plurality of segments, the robotic arm extending from a base;

receiving input data relating to a position of the bronchoscope from a plurality of data sources;

determining the position of the bronchoscope within the bronchial network based on the input data from one or more of the plurality of data sources;

displaying the determined position of the bronchoscope relative to the displayed model of the bronchial network;

displaying navigational guidance comprising: the determined path to the target location, a distance to the target location, a location of the bronchoscope relative to the model of the bronchial network, and depth data.

13. The method of claim 12 , further comprising:

displaying the target location on the displayed model of the bronchial network.

14. The method of claim 13 , further comprising displaying the target location relative to a position and orientation of the bronchoscope.

15. The method of claim 12 , wherein the plurality of data sources comprises at least one of:

position sensor data received from a position sensor on the bronchoscope,

image data received from an imaging device on the bronchoscope,

robot data comprising command data for instructing the system to change the orientation of the bronchoscope, and

model data.

16. The method of claim 15 , wherein the robot data comprises at least one of:

a pitch, a roll, and a yaw of the bronchoscope; and

an insertion distance of the bronchoscope into the bronchial network.

17. The method of claim 15 , further comprising:

generating the computer model based on computerized axial tomography (CT) scans of the tubular network.

18. The method of claim 12 , wherein accessing the determined path from the entry point of the bronchial network to the target location comprises determining a path along centerlines of lumens of the model.

19. The method of claim 12 , further comprising displaying a computer model of the bronchoscope relative to the displayed model of the bronchial network.

20. The method of claim 12 , further comprising receiving a user command from a command console including a control module configured to allow an operator to remotely control the bronchoscope.

21. The method of claim 20 , wherein the control module comprises a trackball.

22. The method of claim 12 , further comprising inserting a biopsy tool or surgical instrument into a working channel of the bronchoscope, wherein the bronchoscope includes an imaging device, and wherein the biopsy tool or surgical instrument is inserted into the working channel without removing the imaging device.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 10, 2019
From: GHOREYSHI, ATIYEH
To: AURIS SURGICAL ROBOTICS, INC.
Reel/Frame 050332/0025 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 10, 2019
From: MINTZ, DAVID S.; JEEVAN, PRASANTH
To: AURIS SURGICAL ROBOTICS, INC.
Reel/Frame 050332/0061 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 10, 2019
From: XU, YILIANG; YANG, GEHUA; LEOTTA, MATTHEW JOSEPH; STEWART, CHARLES V.
To: KITWARE INC.
Reel/Frame 050332/0092 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 10, 2019
From: KITWARE INC.
To: AURIS SURGICAL ROBOTICS, INC.
Reel/Frame 050332/0120 →
CHANGE OF NAME Recorded Sep 10, 2019
From: AURIS SURGICAL ROBOTICS, INC.
To: AURIS HEALTH, INC.
Reel/Frame 050337/0780 →
Continuity (5)
Continuation 16222686 · Dec 17, 2018
Continuation 15669258 · Aug 4, 2017
Continuation 15268238 · Sep 16, 2016
Provisional Application 62220770 · Sep 18, 2015
Related Publication 20190228528A1 · Jul 25, 2019
Cited By (76)
US 1,069,125 US 1,069,126 US 1,094,724 US 1,095,845 US 1,121,033 US 1,121,695 US 12,193,764 US 12,193,769 US 12,208,220 US 12,220,150 US 12,226,168 US 12,226,174 US 12,226,175 US 12,226,176 US 12,232,711 US 12,251,174 US 12,251,178 US 12,256,923 US 12,257,006 US 12,268,460 US 12,290,237 US 12,290,239 US 12,290,277 US 12,295,672 US 12,295,692 US 12,303,220 US 12,310,669 US 12,310,673 US 12,310,804 US 12,311,530 US 12,318,102 US 12,318,137 US 12,324,645 US 12,329,477 US 12,329,485 US 12,343,483 US 12,350,449 US 12,357,405 US 12,357,409 US 12,364,557 US 12,370,002 US 12,376,918 US 12,376,926 US 12,383,352 US 12,390,286 US 12,414,686 US 12,414,823 US 12,420,063 US 12,433,696 US 12,433,706 US 12,440,235 US 12,447,308 US 12,465,431 US 12,472,020 US 12,478,444 US 12,491,042 US 12,502,229 US 12,507,883 US 12,521,277 US 12,544,167 US 12,564,459 US 12,569,307 US 12,575,899 US 12,575,901 US 12,582,485 US 12,588,964 US 12,605,086 US 12,611,266 US 12,616,534 US 12,616,539 US 12,629,002 US 12,642,612 US 12,653,374 US 12,672,793 US 12,702,515 US 12,727,950