System and method for tracking a curved needle

1 . A method for tracking a needle, comprising:

determining a visibility of the needle being inserted into a subject in an image of a sequence of images;

in response to determining that the visibility satisfies a visibility threshold, detecting a location of the needle based on at least one first algorithm and a detected curvature of the needle, wherein detecting the location of the needle based on the at least one first algorithm comprises:

determining a length of a visible portion of the needle while the needle is inserted into the subject; and

detecting the location of the needle based on the length of the visible portion of the needle being inserted into the subject;

determining an updated visibility of the needle being inserted into a subject in an image of a sequence of images;

in response to determining that the updated visibility does not satisfy the visibility threshold, detecting an updated location of the needle being inserted based on at least one second algorithm different than the at least one first algorithm; and

tracking the location of the needle in the sequence of images based on locations detected with the at least one first algorithm and the at least one second algorithm.

2 . The method of claim 1 , wherein detecting the location of the needle based on the at least one first algorithm comprises:

determining a length of a visible portion of the needle being inserted into the subject;

in response to determining that the length satisfies a length threshold, detecting the location of the needle based on a weighted polynomial fitting algorithm and the detected curvature of the needle; and

in response to determining that the length does not satisfy the length threshold, detecting the location of the needle based on a straight needle tracking algorithm.

3 . The method of claim 2 , wherein detecting the location of the needle based on the weighted polynomial fitting algorithm comprises:

segmenting the image by identifying candidate pixels corresponding to the needle based on an intensity threshold;

weighing each pixel of the candidate pixels based on a normalized intensity, a gradient magnitude, and a distance to an estimated location of a tip of the needle; and

fitting a curve to the candidate pixels based on weights of at least a portion of pixels of the candidate pixels.

4 . The method of claim 3 , further comprising:

filtering the image after identifying the candidate pixels and before weighing each pixel; and

identifying additional candidate pixels based on connections to the candidate pixels.

5 . The method of claim 2 , wherein detecting the location of the needle based on the straight needle tracking algorithm comprises:

adjusting a location of a tip of the needle and/or an insertion point of the needle.

6 . The method of claim 5 , wherein adjusting the location of the tip and/or the insertion point comprises:

extending a line corresponding to a visual portion of the needle in a first direction to a side of the image;

extending the line in a second direction opposite the first direction until a length of the line matches a length of the needle in a previous image in the sequence of images; and

identifying the tip of the needle in a range of pixels surrounding an end of the line.

7 . The method of claim 1 , wherein detecting the location of the needle being inserted based on the at least one second algorithm comprises:

calculating a histogram gradient for each cell in the image; and

detecting a deformation of tissue based on the histogram gradient of each cell for the image and a static background image.

8 . The method of claim 1 , wherein detecting the location of the needle being inserted based on the at least one second algorithm comprises:

estimating, with a kinematics tracking algorithm, a location of a tip of the needle based on kinematics data and a compensation factor based on a confidence score.

9 . The method claim 1 , wherein detecting the location of the needle based on the at least one first algorithm and the detected curvature of the needle comprises:

estimating a main axis of a region of interest including the needle.

10 . The method of claim 1 , further comprising:

in response to determining that the visibility satisfies a visibility threshold, updating kinematics information associated with the at least one second algorithm.

11 . A system for tracking a needle, comprising at least one computing device programmed or configured to:

determine a visibility of the needle being inserted into a subject in an image of a sequence of images;

in response to determining that the visibility satisfies a visibility threshold, detect a location of the needle based on at least one first algorithm and a detected curvature of the needle, wherein detecting the location of the needle based on the at least one first algorithm comprises:

determining a length of a visible portion of the needle while the needle is inserted into the subject; and

detecting the location of the needle based on the length of the visible portion of the needle being inserted into the subject;

determining an updated visibility of the needle being inserted into a subject in an image of a sequence of images;

in response to determining that the visibility does not satisfy the visibility threshold, detect the location of the needle being inserted based on at least one second algorithm; and

track the location of the needle in the sequence of images based on locations detected with the at least one first algorithm and/or the at least one second algorithm as continuously or continually determined based on comparing the visibility of the needle to the visibility threshold.

12 . The system of claim 11 , wherein detecting the location of the needle based on the at least one first algorithm comprises:

determining a length of a visible portion of the needle being inserted into the subject;

in response to determining that the length satisfies a length threshold, detecting the location of the needle based on a weighted polynomial fitting algorithm and the detected curvature of the needle; and

in response to determining that the length does not satisfy the length threshold, detecting the location of the needle based on a straight needle tracking algorithm.

13 . The system of claim 12 , wherein detecting the location of the needle based on the weighted polynomial fitting algorithm comprises:

segmenting the image by identifying candidate pixels corresponding to the needle based on an intensity threshold;

weighing each pixel of the candidate pixels based on a normalized intensity, a gradient magnitude, and a distance to an estimated location of a tip of the needle; and

fitting a curve to the candidate pixels based on weights of at least a portion of pixels of the candidate pixels.

14 . The system of claim 13 , the at least one computing device further configured to:

filter the image after identifying the candidate pixels and before weighing each pixel; and

identify additional candidate pixels based on connections to the candidate pixels.

15 . The system of claim 12 , wherein detecting the location of the needle based on the straight needle tracking algorithm comprises:

adjusting a location of a tip of the needle and/or an insertion point of the needle.

16 . The system of claim 15 , wherein adjusting the location of the tip and/or the insertion point comprises:

extending a line corresponding to a visual portion of the needle in a first direction to a side of the image;

extending the line in a second direction opposite the first direction until a length of the line matches a length of the needle in a previous image in the sequence of images; and

identifying the tip of the needle in a range of pixels surrounding an end of the line.

17 . The system of claim 11 , wherein detecting the location of the needle being inserted based on the at least one second algorithm comprises:

calculating a histogram gradient for each cell in the image; and

detecting a deformation of tissue based on the histogram gradient of each cell for the image and a static background image.

18 . The system of claim 11 , wherein detecting the location of the needle being inserted based on the at least one second algorithm comprises:

estimating, with a kinematics tracking algorithm, a location of a tip of the needle based on kinematics data and a compensation factor based on a confidence score.

19 . The system of claim 11 , wherein detecting the location of the needle based on the at least one first algorithm and the detected curvature of the needle comprises:

estimating a main axis of a region of interest including the needle.

20 . A computer program product for tracking a needle, comprising at least one non-transitory computer-readable medium including instructions that, when executed by at least one computing device, cause the at least one computing device to:

determine a visibility of the needle being inserted into a subject in an image of a sequence of images;

in response to determining that the visibility satisfies a visibility threshold, detect a location of the needle based on at least one first algorithm, a length of a visible portion of the needle, and a detected curvature of the needle;

segment the image identifying candidate pixels corresponding to the needle based on the location of the needle, resulting in a segmented image;

after detecting the location of the needle, determine a new visibility of the needle being inserted into the subject in at least a second image of the sequence of images, resulting in a segmented second image;

in response to determining that the new visibility does not satisfy the visibility threshold, detect a new location of the needle being inserted based on at least one second algorithm;

segment the at least a second image based on the new location; and

continue to track the location of the needle in the sequence of images based on the segmented image, the segmented second image, and at least one further segmented image of the sequence of images as continuously or continually determined based on comparing the visibility of the needle to the visibility threshold.