Using deep learning to process images of the eye to predict visual acuity

1 . A method for predicting visual acuity based on image processing, the method comprising:

accessing an image of at least part of an eye of a subject, wherein the image was captured by an imaging device on a first date;

inputting the image into a machine-learning model to determine:

a first predicted visual-acuity metric that predicts a visual acuity of the subject as would be determined based on performing a visual-acuity assessment of the subject on the first date on which the image was captured; and

a second predicted visual-acuity metric that predicts the visual acuity of the subject as would be determined based on performing the visual-acuity assessment of the subject on a second date that is at a specified interval after the first date,

wherein the machine-learning model comprises:

a set of parameters determined using:

a set of training images corresponding to a set of training subjects, wherein each image of the set of training images depicts at least part of an eye of a corresponding training subject of the set of training subjects;

wherein the set of training images comprises a first set of training images and a second set of training images;

and

a set of labels identifying an observed visual acuity metric for each subject of the set of training subjects;

wherein the observed visual acuity metric comprises a numeric visual acuity value and/or an identification of a numerical visual-acuity range; and

wherein the set of labels comprise a first set of labels and a second set of labels;

wherein the first set of labels are associated with the first set of training images and identify the observed visual acuity metric for each subject of the set of training subjects at the time that the image was collected; and

wherein the second set of labels are associated with the second set of training images and identify the observed visual acuity metric for each subject of the set of training subjects at a time after the image was collected;

and

a function relating the image and the set of parameters to the first predicted visual-acuity metric and the second predicted visual-acuity metric;

and

returning the first predicted visual-acuity metric and the second predicted visual acuity metric.

2 . The method of claim 1 , wherein the image is an optical coherence tomography (OCT) image.

3 . The method of claim 1 , wherein the image is a color fundus photograph image.

4 . The method of claim 1 , wherein the second date is 12 months after the first date.

5 . The method of claim 1 , wherein the second predicted visual-acuity metric includes one or more numbers, the one or more numbers representing the second predicted visual acuity on the second date, wherein the second date is at least 6 months after the first date when the image was captured.

6 . The method of claim 1 , wherein the first predicted visual-acuity metric is a binary value that predicts whether the first predicted visual acuity of the subject on the first date was worse than a threshold visual-acuity value.

7 . The method of claim 1 , wherein the second predicted visual-acuity metric is a binary value that predicts whether the second predicted visual acuity of the subject was worse than a threshold visual-acuity value, wherein the second date is at least 6 months after the first date.

8 . The method of claim 7 wherein the threshold visual-acuity value is a Snellen fraction of 20/160, 20/80 or 20/40.

9 . The method of claim 1 , further comprising, responsive to the first predicted visual-acuity metric and the second predicted visual-acuity metric: providing a recommendation that the subject receive a pharmacological treatment.

10 . The method of claim 1 , wherein the image is a three-dimensional image, and wherein the method further comprises:

slicing the image into a plurality of two-dimensional slices, each slice of the plurality of two-dimensional slices captured at a different scan-centered angle and offset a different number of pixels from each other slice; and

wherein inputting the image into the machine-learning model comprises inputting the plurality of two-dimensional slices into the machine-learning model.

11 . The method of claim 1 , wherein the machine-learning model is a deep learning model.

12 . The method of claim 1 , wherein the machine-learning model is or includes a convolutional neural network.

13 . The method of claim 1 , wherein the machine-learning model uses a set of convolution kernels.

14 . The method of claim 1 , wherein the machine-learning model includes at least one of a ResNet model or an Inception model.

15 . The method of claim 1 , wherein the visual-acuity assessment evaluates a response of the subject to one or more visual stimuli.

16 . A method of stratifying a clinical study, the method comprising:

performing, for each subject of a set of subjects, the method for predicting visual acuity based on image processing of claim 1 ;

determining, for each of the set of subjects, whether the subject is eligible to participate in a clinical study, wherein the determination is based on whether each of a set of eligibility criteria is satisfied in relation to the subject, and wherein evaluation of a particular eligibility criterion of the set of eligibility criteria is performed using at least one of the first predicted visual-acuity metric and the second predicted visual-acuity metric for the subject; and

wherein the clinical study is determined to be performed with a subset of the set of subjects, wherein each subject in the subset was determined to be eligible to participate in the clinical study.

17 . The method of claim 16 , wherein the clinical study is determined to be conducted in accordance with subject assignments to first and second subject groups.

18 . The method of claim 1 , wherein the observed visual acuity metric comprises the numeric visual acuity value.

19 . The method of claim 1 , further comprising returning a predicted rate of decline of visual acuity of the subject.

20 . A system comprising:

one or more data processors; and

a non-transitory computer readable storage medium containing instructions which, when executed on the one or more data processors, cause the one or more data processors to:

access an image of at least part of an eye of a subject, wherein the image was captured by an imaging device on a first date;

input the image into a machine-learning model to determine:

a first predicted visual-acuity metric that predicts a visual acuity of the subject as would be determined based on performing a visual-acuity assessment of the subject on the first date on which the image was captured; and

a second predicted visual-acuity metric that predicts the visual acuity of the subject as would be determined based on performing the visual-acuity assessment of the subject on a second date that is at a specified interval after the first date,

wherein the machine-learning model comprises:

a set of parameters determined using:

a set of training images corresponding to a set of training subjects, wherein each image of the set of training images depicts at least part of an eye of a corresponding training subject of the set of training subjects;

 wherein the set of training images comprises a first set of training images and a second set of training images;

and

a set of labels identifying an observed visual acuity metric for each subject of the set of training subjects;

 wherein the observed visual acuity metric comprises a numeric visual acuity value and/or an identification of a numerical visual-acuity range; and

 wherein the set of labels comprise a first set of labels and a second set of labels;

 wherein the first set of labels are associated with the first set of training images and identify the observed visual acuity metric for each subject of the set of training subjects at the time that the image was collected; and

 wherein the second set of labels are associated with the second set of training images and identify the observed visual acuity metric for each subject of the set of training subjects at a time after the image was collected;

and

a function relating the image and the set of parameters to the first predicted visual-acuity metric and the second predicted visual-acuity metric; and

return the first predicted visual-acuity metric and the second predicted visual-acuity metric.

21 . The system of claim 20 , wherein the observed visual acuity metric comprises the numeric visual acuity value.