Granted Patent B2

US 12,478,444 · App. 16/826,114 · Granted Nov 25, 2025

Inventors: David B. Camarillo (Aptos, CA); David Eng (Saratoga, CA); Jake Sganga (Laguna Beach, CA)

Assignee: The Board of Trustees of the Leland Stanford Junior University

A61B34/70A61B1/2676A61B34/10A61B34/20A61B34/37G06N3/02G16H30/00G16H50/00A61B2034/102A61B2034/301

Abstract

Claims (62)

1 . A system for localization within a luminal network, the system comprising:

an instrument comprising:

an elongate body configured to be inserted into the luminal network, and

an imaging device positioned on a distal portion of the elongate body;

at least one computer-readable memory having stored thereon executable instructions; and

one or more processors in communication with the at least one computer-readable memory and configured to execute the instructions to cause the system to at least:

receive, from the imaging device, image data comprising an image captured when the elongate body is within the luminal network, the image depicting one or more branchings of the luminal network;

access a machine learning model configured to:

identify the one or more branchings from the image data;

compare the one or more branchings from the image data with one or more virtual branchings in a repository to determine a match;

determine a location within the luminal network corresponding to the match; and

output a position of the distal portion of the instrument; and

determine, based on the position of the distal portion of the instrument output from the machine learning model, a location of the distal portion of the elongate body within the luminal network.

2 . The system of claim 1 , wherein the machine learning model comprises a convolutional neural network trained to identify branchings within the one or more luminal networks.

3 . The system of claim 1 , wherein the machine learning model is based at least in part on simulated imagery of the luminal network, the simulated imagery comprising at least one computer-generated image of the luminal network over at least three dimensions, the simulated imagery depicting a simulated perspective from within the luminal network.

4 . The system of claim 1 , wherein the image data comprises video imagery from a bronchial network of a patient.

5 . The system of claim 1 , further comprising a robotic arm configured to, in use, navigate the elongate body and imaging device through the luminal network, wherein the one or more processors are further configured to:

access information regarding a target location within the luminal network; and

navigate, based on the determined location of the distal portion of the elongate body, the elongate body and the imaging device toward the target location.

6 . The system of claim 1 , wherein determining the location of the distal portion of the elongate body comprises determining a position relative to a center line of a portion of the luminal network.

7 . The system of claim 1 , wherein determining the location of the distal portion of the elongate body comprises determining, relative to a fixed reference point, three or more of:

a position along a first axis;

a position along a second axis orthogonal to the first axis;

a position along a third axis orthogonal to both the first and second axis;

a rotational orientation about the first axis;

a rotational orientation about the second axis; and

a rotational orientation about the third axis.

8 . The system of claim 1 , wherein the one or more processors are further configured to automatically transmit instructions to display:

a first display interface comprising a first set of one or more images from a perspective of the imaging device, the first set of images from the perspective of the imaging device comprising one or more images captured by the imaging device;

a second display interface comprising a second set of one or more images from a global-based perspective, the second set of images based on a three-dimensional ( 3 D) scan of the luminal network; and

an indication on the second display interface of the location of the distal portion of the elongate body.

9 . The system of claim 1 , wherein:

the machine learning model is configured to identify the one or more branchings of the luminal network from the image data,

accessing the machine learning model comprises predicting, based on a particle filter, one or more features within the image data,

the one or more processors are further configured to determine probabilities, based on the particle filter and the one or more features, that the one or more branchings identified by the machine learning model matches with one or more corresponding branchings in a preoperative model of the luminal network, and

determining the location of the distal portion of the elongate body within the luminal network comprises determining the location based on the probabilities.

10 . The system of claim 1 , wherein the image data is camera image and the position of the distal portion of the instrument output from the machine learning model includes a location and a direction of the one or more branchings.

11 . The system of claim 1 , wherein the image data is received in real-time and the machine learning model outputs the position of the distal portion of the instrument in real-time.

12 . A system for localization within a luminal network, the system comprising:

an instrument comprising:

an elongate body configured to be inserted into the luminal network, and

an imaging device positioned on a distal portion of the elongate body;

at least one computer-readable memory having stored thereon executable instructions; and

one or more processors in communication with the at least one computer-readable memory and configured to execute the instructions to cause the system to at least:

receive, from the imaging device, image data comprising an image captured when the elongate body is within the luminal network, the image depicting one or more branchings of the luminal network;

access a machine learning model, wherein the machine learning model is configured to identify the one or more branchings of the luminal network from the image data;

predicting, based on a particle filter, one or more features within the image data,

determining probabilities, based on the particle filter and the one or more features, that the one or more branchings identified by the machine learning model matches with one or more corresponding branchings in a preoperative model of the luminal network; and

determine, based on the machine learning model and information regarding the one or more branchings probabilities, a location of the distal portion of the elongate body within the luminal network.

13 . A system for localization within a luminal network, the system comprising:

an instrument comprising:

an elongate body configured to be inserted into the luminal network, and

an imaging device positioned on a distal portion of the elongate body;

at least one computer-readable memory having stored thereon executable instructions; and

one or more processors in communication with the at least one computer-readable memory and configured to execute the instructions to cause the system to at least:

receive, from the imaging device, image data comprising an image captured when the elongate body is within the luminal network;

access a machine learning model configured to:

extract one or more features from the image data;

compare the one or more features from the image data with one or more virtual features in a repository to determine a match;

determine a location within the luminal network corresponding to the match; and

output a position of the distal portion of the instrument; and

determine, based on the position of the distal portion of the instrument output from the machine learning model, a location of the distal portion of the elongate body within the luminal network.

Assignments (1)

Continuity (2)

References Cited (400)