Method and system for providing visual guidance to an operator for steering a tip of an endoscopic device toward one or more landmarks in a patient

1. A computer implemented method for providing guidance to steer an endoscopic device towards one or more landmarks in a patient, the method comprising:

determining a current position of a reference point on the endoscopic device relative to a reference frame;

determining a first vector from the current position of the reference point to a position of a first landmark relative to the reference frame;

transforming the first vector from the reference frame to an image reference frame that corresponds to the perspective of an image capturing device disposed at a tip of the endoscopic device; and

displaying a graphical representation of the transformed first vector along with and positioned relative to a current image captured by the image capturing device on a display screen so as to indicate a direction to steer the tip of the endoscopic device towards the first landmark.

2. The method according to claim 1 , wherein one or more optical fibers configured with strain sensors extend through the endoscopic device to provide position and shape information of the endoscopic device.

3. The method according to claim 1 , wherein the current image is a stereoscopic image, the display screen is a three-dimensional display screen, and the graphical representation of the transformed first vector is a three-dimensional arrow pointing towards the direction to steer the tip of the endoscopic device towards the first landmark.

4. The method according to claim 1 , further comprising prior to determining the first vector:

receiving an indication to establish the first landmark;

determining a position of the reference point at the time of receiving the indication to establish the first landmark;

storing the position of the reference point at the time of receiving the indication to establish the first landmark in a memory device; and

referencing the stored position as the position of the first landmark.

5. The method according to claim 4 , further comprising prior to determining the first vector:

capturing an image of the first landmark at a first opportunity after receiving the indication to establish the first landmark; and

storing the captured image of the first landmark in the memory device;

wherein the displaying of the graphical representation of the transformed first vector along with the current image on the display screen further includes displaying the captured image of the first landmark on the display screen proximate to the graphical representation of the transformed first vector.

6. The method according to claim 1 , further comprising:

determining a second vector from the current position of the reference point to a position of a second landmark relative to the reference frame;

transforming the second vector from the reference frame to the image reference frame; and

displaying a graphical representation of the transformed second vector along with the current image on the display screen so as to indicate a direction to steer the tip of the endoscopic device towards the second landmark.

7. The method according to claim 1 , further comprising:

periodically repeating the determination of the current reference point position, the determination of the first vector, the transformation of the first vector, and the display of the graphical representation of the transformed first vector along with the current image on the display screen until an indication to stop such periodically repeated actions is received.

8. The method according to claim 1 , wherein a size of the graphical representation of the transformed first vector is related to the magnitude of the transformed first vector.

9. The method according to claim 1 , wherein a color of the graphical representation of the transformed first vector is a color associated with the first landmark.

10. The method according to claim 1 , wherein the displaying of the graphical representation of the transformed first vector comprises displaying the graphical representation adjacent a side of the displayed current image that is indicative of the direction to steer the tip of the endoscopic device towards the first landmark.

11. The method according to claim 1 , further comprising:

storing positions of a plurality of landmarks including the position of the first landmark in a memory device prior to determining vectors from the current position of the reference point to the positions of the plurality of landmarks.

12. The method according to claim 11 , wherein the positions of the plurality of landmarks are based upon preoperative measured data.

13. The method according to claim 1 , further comprising: displaying computer models of the patient and the first landmark on a second display screen.

14. The method according to claim 13 , wherein a color of the graphical representation of the transformed first vector is a same color as the computer model of the first landmark being displayed on the second display screen.

15. The method according to claim 1 , further comprising: displaying computer models of the patient and the first landmark on the display screen.

16. The method according to claim 15 , wherein a color of the graphical representation of the transformed first vector is a same color as the computer model of the first landmark being displayed on the display screen.