Systems and methods for detecting an orientation of medical instruments
A method for determining a position of a tool being received by a catheter, the method including capturing first images with the tool as the tool is being installed in the catheter. The method also includes generating training images for a deep convolutional neural network (DCNN) by replicating the first images and applying perturbations to the replicated first images. The method also includes training the DCNN by inputting the training images into the DCNN. The method also includes capturing second images with the tool as the tool is being installed in the catheter. The method also includes inputting the second images into the DCNN. The method also includes analyzing the second images with the trained DCNN. The method further includes determining a configuration of the tool based on the analyzed second images.
1. A method for determining a position of a tool being received by a catheter, the method comprising:
capturing first images of the catheter with the tool as the tool is being installed in the catheter;
inputting the first images into a deep convolutional neural network (DCNN) trained using training images generated by replicating and applying perturbations to second images from one or more second tools captured as the one or more second tools are being installed in one or more second catheters;
analyzing the first images with the DCNN; and
determining a configuration of the tool based on the analyzing of the first images.
2. The method of claim 1 , wherein the determining the configuration includes determining that a position of the tool is one of outside the catheter, inside the catheter and not extended into a patient anatomy, or inside the catheter and extended into the patient anatomy.
3. The method of claim 1 , wherein the determining the configuration includes:
determining an angular orientation of a feature in the first images; and
determining a rotational offset of the tool relative to the catheter based on the angular orientation of the feature.
4. The method of claim 3 , further comprising:
rotating the first images by the rotational offset; and
displaying the rotated first images on a display unit.
5. The method of claim 3 , further comprising:
receiving manipulation instructions for manipulating a set of control cables to articulate a distal portion of the catheter;
determining adjusted manipulation instructions for manipulating the set of control cables based on the rotational offset; and
applying the adjusted manipulation instructions to articulate the distal portion of the catheter.
6. The method of claim 3 , wherein a distal portion of the tool has a square cross-section and a lumen of the catheter has a complimentary square cross-section, and wherein the rotational offset is selected from a group consisting of 0 (zero), 90, 180, and 270 degrees.
7. The method of claim 3 , wherein the tool has a circular cross-section and a lumen of the catheter has a complimentary circular cross-section, and wherein the rotational offset is within a range of 0 to 360 degrees.
8. The method of claim 3 , wherein the feature is a longitudinal marking that extends longitudinally along an interior wall of the catheter and wherein the second images used to train the DCNN depict the longitudinal marking.
9. The method of claim 1 , further comprising:
capturing the second images with the one or more second tools as the one or more second tools are being installed in the one or more second catheters;
generating the training images for the DCNN by replicating the second images and applying perturbations to the replicated second images, wherein applying perturbations to the replicated second images includes applying rotations to the replicated second images; and
training the DCNN, before capturing the first images, by inputting the training images into the DCNN.
10. The method of claim 1 , further comprising:
capturing the second images with the one or more second tools as the one or more second tools are being installed in the one or more second catheters;
generating the training images for the DCNN by replicating the second images and applying perturbations to the replicated second images, wherein applying perturbations to the replicated second images includes applying color variances to the replicated second images; and
training the DCNN, before capturing the first images, by inputting the training images into the DCNN.
11. A system comprising:
a catheter sized to receive an imaging tool; and
one or more processors configured to:
receive first images of the catheter from the tool as the tool is being inserted in the catheter;
input the first images into a deep convolutional neural network (DCNN) trained using training images generated by replicating and applying perturbations to second images from one or more second tools captured as the one or more second tools are being installed in one or more second catheters;
analyze the first images with the DCNN; and
determine a configuration of the tool based on the analyzing of the first images.
12. The system of claim 11 , wherein determining the configuration includes a determination that a position of the tool is one of outside the catheter, inside the catheter and not extended into a patient anatomy, or inside the catheter and extended into the patient anatomy.
13. The system of claim 11 , wherein determining the configuration includes:
determination of an angular orientation of a feature in the first images; and
determination of a rotational offset of the tool relative to the catheter based on the angular orientation of the feature.
14. The system of claim 13 , wherein the one or more processors are further configured to:
rotate the first images by the rotational offset; and
display the rotated first images on a display unit.
15. The system of claim 13 , wherein the one or more processors are further configured to:
receive manipulation instructions for manipulating a set of control cables to articulate a distal portion of the catheter;
determine adjusted manipulation instructions for manipulating the set of control cables based on the rotational offset; and
apply the adjusted manipulation instructions to articulate the distal portion of the catheter.
16. The system of claim 13 , wherein a distal portion of the tool has a square cross-section and a lumen of the catheter has a complimentary square cross-section, and wherein the rotational offset is selected from a group consisting of 0 (zero), 90, 180, and 270 degrees.
17. The system of claim 13 , wherein the tool has a circular cross-section and a lumen of the catheter has a complimentary circular cross-section, and wherein the rotational offset is within a range of 0 to 360 degrees.
18. The system of claim 13 , wherein the feature is a longitudinal marking that extends longitudinally along an interior wall of the catheter and wherein the training images depict the longitudinal marking.
19. The system of claim 11 , wherein the one or more processors are further configured to:
receive the second images from the one or more second tools as the one or more second tools are being inserted in the one or more second catheters;
generate the training images for the DCNN by replicating the second images and applying perturbations to the replicated second images, wherein applying perturbations to the replicated second images includes applying rotations to the replicated second images; and
train the DCNN, before capturing the first images, by inputting the training images into the DCNN.
20. The system of claim 11 , wherein the one or more processors are further configured to:
receive the second images from the one or more second tools as the one or more second tools are being inserted in the one or more second catheters;
generate the training images for the DCNN by replicating the second images and applying perturbations to the replicated second images, wherein applying perturbations to the replicated second images includes applying color variances to the replicated second images; and
train the DCNN, before capturing the first images, by inputting the training images into the DCNN.
21. The system of claim 11 , wherein the one or more second catheters include the catheter and the one or more second tools include the tool.
22. The method of claim 1 , wherein the one or more second catheters include the catheter and the one or more second tools include the tool.