Methods and systems for identification of trabecular meshwork or other intraoccular anatomical structures and tissue types

1 . A system for identifying an anatomical structure or tissue type within an eye of a subject, said system comprising:

a viewing device for viewing an area within the eye which includes the anatomical structure or tissue type;

a camera or imaging device;

an image processor; and

a display device;

wherein the camera or imaging device acquires an image of the area viewed by the viewing device;

wherein the image processor uses deep learning to identify the anatomical structure or tissue type on said image; and

wherein the display device displays an enhanced image on which the anatomical structure or tissue type is marked;

wherein the image processor comprises a non-transitory data storage medium having recorded thereon a deep-learning generative model that outputs said enhanced image on which the anatomical structure or tissue type is marked;

wherein the deep-learning generative model comprises at least one Convolutional Neural Network (CNN) that has been trained using data augmentation; and

wherein any pixels with missing values after augmentation were populated by replicating the closest pixel values of the image.

2 . A system according to claim 1 wherein the viewing device is selected from: goniolens, gonioprism, gonioscope, retnascope, ophthalmoscope and surgical microscope.

3 . A system according to claim 1 wherein the camera or imaging device comprises a still camera or a video camera.

4 . A system according to claim 1 wherein the image processor comprises a computer or processor.

5 . A system according to claim 1 wherein the display device is selected from: a video screen, monocular or binocular eyepiece, virtual reality headset/goggles/glasses.

6 . A system according to claim 1 wherein the anatomical structure or tissue type is selected from: trabecular meshwork, macula, retinal pigment epithelium, optic nerve head, epiretinal membranes.

7 . A system according to claim 1 wherein the viewing device views the iridocorneal angle and the anatomical structure or tissue type is trabecular meshwork.

8 . A system according to claim 1 wherein the anatomical structure or tissue type is marked on the enhanced image by a type of marking selected from: delineation, accentuation, darkening, coloring, annotation or indication with an arrow or pointer.

9 . A system according to claim 1 wherein;

said at least one Convolutional Neural Network (CNN) comprises a plurality of Neural Encoder-Decoder Convolutional Neural Networks (U-nets) trained to identify the anatomical structure or tissue type, and the image processor is programmed to apply said Neural Encoder-Decoder Convolutional Neural Networks CNNs-(U-nets).

10 . A system according to claim 1 wherein the system is equipped to transmit the enhanced image to a local or remote secondary location.

11 . A system according to claim 10 wherein the secondary location is selected from: a local or cloud-based data storage medium, hard drive, intraoperative recording device, remote computer, remote display, telemedicine terminal, medical records database or surgical robot.

12 . A system according to claim 1 wherein said at least one Convolutional Neural Network (CNN) is configured as an encoder and decoder network that is able to generate predictions for every pixel in said image.

13 . A system according to claim 12 wherein said at least one Convolutional Neural Network (CNN) is trained using video image frames that are down-sampled to a size of 512×288 pixels.

14 . A system according to claim 13 wherein said at least one Convolutional Neural Network (CNN) is trained by a method which uses multitask learning, whereby said at least one Convolutional Neural Network the (CNN) is trained to predict not only the target anatomical structure or tissue type but also an ancillary anatomical structure or tissue type.

15 . A system according to claim 14 wherein:

the target anatomical structure or tissue type is trabecular meshwork; and

the ancillary anatomical structure or tissue type is iris border.

16 . A system according to claim 1 wherein the data augmentation comprises at least one of:

horizontal mirroring;

random translations of images up to ±10% of the image height;

rotations up to ±25 degrees;

shearing up to ±20 degrees;

zooming up to ±10% of the image height; and

small random changes to the color balance of any red/green/blue channels of an image.

17 . A system for identifying an anatomical structure or tissue type within an eye of a subject, said system comprising:

a viewing device for viewing an area within the eye which includes the anatomical structure or tissue type;

a camera or imaging device;

an image processor; and

a display device;

wherein the camera or imaging device acquires an image of the area viewed by the viewing device;

wherein the image processor uses deep learning to identify the anatomical structure or tissue type on said image;

wherein the image processor comprises a non-transitory data storage medium having recorded thereon a deep-learning generative model that outputs said enhanced image on which the anatomical structure or tissue type is marked; and

wherein the deep-learning generative model comprises one or more Convolutional Neural Networks (CNNs) and the display device displays an enhanced image on which the anatomical structure or tissue type is marked and wherein cross-validation was applied during training of the one or more CNNs.

18 . A system according to claim 17 wherein the cross-validation comprises per-patient stratified ten-fold cross-validation.

19 . A system according to claim 17 wherein the anatomical structure or tissue type is selected from: trabecular meshwork, macula, retinal pigment epithelium, optic nerve head, epiretinal membranes.

20 . A system according to claim 17 wherein the system is equipped to transmit the enhanced image to a local or remote secondary location selected from: a local or cloud-based data storage medium, hard drive, intraoperative recording device, remote computer, remote display, telemedicine terminal, medical records database or surgical robot.