Machine learning for detection of diseases from external anterior eye images
The present disclosure is directed to systems and methods that leverage machine learning for detection of eye or non-eye (e.g., systemic) diseases from external anterior eye images. In particular, a computing system can include and use one or more machine-learned disease detection models to provide disease predictions for a patient based on external anterior eye images of the patient. Specifically, in some example implementations, a computing system can obtain one or more external images that depict an anterior portion of an eye of a patient. The computing system can process the one or more external images with the one or more machine-learned disease detection models to generate a disease prediction for the patient relative to one or more diseases, including, as examples, diseases which present manifestations in a posterior of the eye (e.g., diabetic retinopathy) or systemic diseases (e.g., poorly controlled diabetes).
1 . A computing system for detection of diseases from external anterior eye images, the computing system comprising:
one or more processors; and
one or more non-transitory computer-readable media that collectively store:
one or more machine-learned disease detection models configured to provide disease predictions based on external anterior eye images; and
instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
obtaining one or more external images that depict an anterior portion of an eye of a patient;
processing the one or more external images with the one or more machine-learned disease detection models to generate a disease prediction for the patient relative to one or more diseases,
wherein the one or more diseases comprise one or more disease manifestations in the eye, and
wherein the one or more disease manifestations in the eye comprise one or more disease manifestations in a fundus of the eye; and
providing the disease prediction for the patient relative to the one or more diseases as an output.
2 . The computing system of claim 1 , wherein the one or more diseases comprise one or more systemic diseases.
3 . The computing system of claim 2 , wherein the one or more systemic diseases comprise a blood sugar control disease.
4 . The computing system of claim 2 , wherein the one or more systemic diseases comprise diabetes.
5 . The computing system of claim 2 , wherein the one or more systemic diseases comprise cardiovascular risk or adverse cardiac outcomes, hypertension, anemia, chronic kidney disease, sleep apnea, hypercholesterolemia/atherosclerosis, thyroid disease, hyperparathyroidism, chronic renal failure, hyperlipidemia, or gout.
6 . The computing system of claim 1 , wherein the one or more disease manifestations in the fundus of the eye comprise diabetic retinopathy.
7 . The computing system of claim 1 , wherein the one or more disease manifestations in the fundus of the eye comprise diabetic macular edema.
8 . The computing system of claim 1 , wherein the one or more disease manifestations in the fundus of the eye comprise one or more microaneurysms in the fundus of the eye.
9 . The computing system of claim 1 , wherein the one or more disease manifestations in the fundus of the eye comprise glaucoma or age-related macular degeneration.
10 . The computing system of claim 1 , wherein the disease prediction for the patient relative to the one or more diseases comprises a predicted hemoglobin A1c level for the patient.
11 . The computing system of claim 1 , wherein the disease prediction for the patient relative to the one or more diseases comprises one or more predicted probabilities that the patient is respectively experiencing the one or more diseases.
12 . The computing system of claim 1 , wherein the disease prediction for the patient relative to the one or more diseases comprises one or more predicted severity levels respectively for the one or more diseases.
13 . The computing system of claim 1 , wherein the one or more external images comprise one or more images captured by a user device.
14 . The computing system of claim 13 , wherein the user device was operated by the patient at a time of capture of the one or more images captured by the user device.
15 . The computing system of claim 13 , wherein the user device comprises a camera of a laptop or a camera of a smartphone or tablet.
16 . The computing system of claim 15 , wherein the camera comprises a front facing camera positioned on a same side of the smartphone or tablet as a display that depicts a viewfinder for the camera.
17 . The computing system of claim 13 , wherein the one or more external images comprise one or more images collected by a slit lamp camera.
18 . The computing system of claim 13 , wherein the one or more external images comprise one or more images collected by a fundoscopic camera.
19 . The computing system of claim 13 , wherein the one or more machine-learned disease detection models comprise one or more convolutional or recurrent neural networks.
20 . The computing system of claim 13 , wherein the one or more machine-learned disease detection models comprise one or more multi-headed neural networks that each have a plurality of heads that respectively output a plurality of predictions.
21 . The computing system of claim 20 , wherein at least a subset of the plurality of heads that respectively output the plurality of predictions provide a plurality of disease predictions respectively for a plurality of different and distinct diseases.
22 . The computing system of claim 21 , wherein at least a subset of the plurality of heads that respectively output the plurality of predictions provide a plurality of severity classification predictions respectively for a plurality of different levels of severity of a single disease.
23 . The computing system of claim 22 , wherein the plurality of severity classification predictions respectively for the plurality of different levels of severity of the single disease comprise five classification predictions respectively for five levels of an International Clinical Diabetic Retinopathy Disease Severity Scale for diabetic retinopathy.
24 . The computing system of claim 22 , wherein the plurality of severity classification predictions respectively for the plurality of different levels of severity of the single disease comprise a plurality of classification predictions respectively for a plurality of ranges of hemoglobin Alc.
25 . The computing system of claim 1 , wherein obtaining the one or more external images comprises providing a graphical, auditory, or tactile user feedback that assists the patient in aligning the anterior portion of the eye with a camera.
26 . The computing system of claim 1 , wherein obtaining the one or more external images comprises detecting and cropping from one or more larger images, image portions that depict anterior portion of the eye of the patient.
27 . The computing system of claim 1 , wherein the one or more external images comprise one or more images having a resolution of 200×200 pixels or less.
28 . A computer-implemented method for training a machine-learned disease detection model to provide disease predictions based on external anterior eye images, the method comprising:
obtaining one or more external images that depict an anterior portion of an eye of a patient, wherein one or more ground truth disease labels are associated with the one or more external images;
processing the one or more external images with one or more machine-learned disease detection models to generate a disease prediction for the patient relative to one or more diseases, wherein the one or more diseases comprise one or more disease manifestations in the eye, and wherein the one or more disease manifestations in the eye comprise one or more disease manifestations in a fundus of the eye;
evaluating a loss function that compares the disease prediction for the patient with the one or more ground truth disease labels; and
modifying one or more values of one or more parameters of the machine-learned disease detection models based at least in part on the loss function.
29 . One or more non-transitory computer-readable media that collectively store:
one or more machine-learned disease detection models configured to provide disease predictions based on external anterior eye images; and
instructions that, when executed by one or more processors, cause a computing system to perform operations, the operations comprising:
obtaining one or more external images that depict an anterior portion of an eye of a patient;
processing the one or more external images with the one or more machine-learned disease detection models to generate a disease prediction for the patient relative to one or more diseases,
wherein the one or more diseases comprise one or more disease manifestations in the eye, and
wherein the one or more disease manifestations in the eye comprise one or more disease manifestations in a fundus of the eye; and
providing the disease prediction for the patient relative to the one or more diseases as an output.