System and method for improved electronic assisted medical procedures

1. A method for displaying an overlay superimposed with an intraoperative image of a surgical field in a medical ophthalmic procedure, the method comprising:

receiving, by a computing device, the overlay for superimposing with the intraoperative image that is in real-time;

receiving, by a computing device, the intraoperative image of the surgical field in the medical ophthalmic procedure, the surgical field including an element that is to appear as above the overlay, said element comprising at least one of: a tool, an iris, a sclera, a corneal reflection, an air bubble, a part of a hand, and a lens fragment, wherein at least one portion of the intraoperative image is associated with the element; and

displaying, by the computing device, the overlay superimposed with the intraoperative image such that an appearance of the overlay in the at least one portion is different than the appearance of the overlay outside of the at least one portion and the overlay appears as below the element and occluded by the element.

2. The method of claim 1 , wherein the overlay is determined based on a preoperative image of an eye, a preoperative OCT, the intraoperative image, a previous intraoperative image, an intraoperative OCT, data from a diagnostic device, data from a phaco-vitrectomy system, a user input, or any combination thereof.

3. The method of claim 1 , wherein the overlay comprises:

a guidance overlay for an intraocular lens (IOL) placement, a guidance overlay for a capsulotomy, a guidance overlay for an incision, an enhancement of a membrane or part of a membrane, an enhancement of a capsule or part of a capsule, symbolic overlay, textual overlay, pictorial overlay, or any combination thereof.

4. The method of claim 1 , wherein the overlay is superimposed such that inside of the at least one portion the overlay is not superimposed and outside of the at least one portion the overlay is superimposed.

5. The method of claim 1 , wherein the displayed overlay appearances in the at least one portion versus outside of the at least one portion differ by at least one of: the overlay transparency, the overlay brightness, the overlay pattern, and the overlay color.

6. The method of claim 1 , further comprising classifying, by the computing device, a pixel of the intraoperative image as being in the at least one portion based on one of the intraoperative image, a previous intraoperative image, or any combination thereof.

7. The method of claim 6 , wherein at least one of the intraoperative image and the previous intraoperative image is segmented, and wherein the classification of a pixel of the intraoperative image as being in the at least one portion is based on the segmentation.

8. The method of claim 7 , further comprising classifying the segments as one of a: pupil, iris, sclera, tool, corneal reflection, air bubble, part of a hand, lens fragment, IOL, or IOL reflections, and determining whether the classified segments should occlude the overlay, based on a type of overlay and predefined rules.

9. The method of claim 6 , wherein the intraoperative image is one of an intraoperative stereoscopic image pair and the classifying of the pixel is based on the pixel disparity calculated from the stereoscopic image pair.

10. The method of claim 6 , wherein the classification is based on a gray level value or color value of at least one of the pixels of the intraoperative image, a neighboring pixel, or any combination thereof.

11. The method of claim 1 , further comprising classifying, by the computing device, a pixel of the intraoperative image as being in the at least one portion based on an intraoperative OCT.

12. The method of claim 11 , wherein data generated by the intraoperative OCT are associated with the pixel of the intraoperative image based on alignment between the intraoperative OCT and a camera that generated the intraoperative image.

13. The method of claim 1 , wherein the element is a tool and further comprising, classifying by the computing device, a pixel as being in the at least one portion based at least on a tool tracker.

14. The method of claim 13 , wherein the classifying is based on a relative position and orientation between the tool and the camera that generated the intraoperative image.