Machine learning system and method for predicting Alzheimer's disease based on retinal fundus images
View Patent ↗A machine learning system and method are disclosed that enable full automation of the process of analyzing retinal fundus images to predict Alzheimer's disease, thereby obviating the need for manual labeling of retinal features while also improving prediction accuracy. A machine learning system and method are disclosed that classify retinal features and predict, based on the classified retinal features, the onset or presence of Alzheimer's disease in a human subject. The system comprises a processor configured to perform one or more machine learning models and a memory device in communication with the processor. The machine learning model(s) is trained to process retinal fundus images acquired by an image acquisition system to classify retinal features contained in the images and to predict, based on the classified retinal features, whether the images are indicative of the presence or onset of Alzheimer's disease.
1 . A machine learning system that classifies retinal features and predicts, based on the classified retinal features, an onset or presence of Alzheimer's disease in a human subject, the system comprising:
a processor configured to perform at least one trained machine learning model configured in a multiple-stage (multi-stage) pipeline architecture comprising at least first, second, and third stages in series, the second stage following the first stage and the third stage following the second stage, the first, second, and third stages comprising, respectively, a trained image quality selector machine learning model, a trained vessel map generator machine learning model and a trained Alzheimer disease classifier machine learning model, wherein each of the at least first, second, and third stages are separately trained, said at least one trained machine learning model has been trained on stored retinal fundus images obtained from at least a first group of human subjects who have previously been diagnosed as having Alzheimer's disease and at least a first group of human subjects who have not previously been diagnosed as having Alzheimer's disease, said at least one trained machine learning model performing a process comprising:
receiving a retinal fundus image that has been acquired by an image acquisition system in a clinical setting;
processing the acquired retinal fundus image at a pixel level to classify one or more retinal features contained in the acquired retinal fundus image, where the trained vessel map generator machine learning model generates a respective vessel map from the retinal fundus image via vessel segmentation; and
predicting, based on the classification, whether the retinal fundus image is indicative of an onset or presence of Alzheimer's disease in the human subject, where the trained Alzheimer disease classifier machine learning model processes the respective vessel map to predict whether the retinal fundus image is indicative of the onset or presence of Alzheimer's disease, and where a treatment strategy is selected based on the prediction and communicated for implementation in the clinical setting and treatment of the human subject is initiated based upon the treatment strategy; and
a memory device in communication with the processor.
2 . The machine learning system of claim 1 , wherein said at least one trained machine learning model is implemented in computer instructions stored in the memory device for execution by the processor.
3 . The machine learning system of claim 1 , wherein the trained image quality selector machine learning model is used during training of the Alzheimer disease classifier machine learning model to classify retinal fundus images inputted to the first stage as being either of sufficient image quality or insufficient image quality based upon image composition, exposure/contrast, artifacts, and sharpness/focus and to output retinal fundus images classified as being of sufficient image quality to the second stage, and wherein during training of the Alzheimer disease classifier machine learning model, the trained vessel map generator machine learning model generates respective vessel maps for each respective retinal fundus image inputted to the second stage and outputs the vessel maps to the third stage, and wherein during training of the Alzheimer disease classifier machine learning model, the vessel maps outputted to the third stage are used to train the Alzheimer's disease classifier model.
4 . The machine learning system of claim 1 , wherein after the Alzheimer disease classifier machine learning model has been trained to classify retinal fundus images as being from a human subject having Alzheimer's disease, a retinal fundus image obtained from a patient is processed by the trained vessel map generator machine learning model to produce the respective vessel map that is outputted to the third stage, and wherein the respective vessel map is processed by the trained Alzheimer disease classifier machine learning model to predict, based on the classification, whether the retinal fundus image is indicative of an onset or presence of Alzheimer's disease in the human subject.
5 . The machine learning system of claim 4 , further comprising a saliency map generator in the third stage for generating respective saliency maps from respective vessel maps generated by the trained vessel map generator learning machine model, where the saliency maps indicate vessel map pixels that are salient to classification to Alzheimer's disease and vessel map pixels that do not significantly contribute to the classification of Alzheimer's disease.
6 . The machine learning system of claim 1 , wherein the trained image quality selector machine learning model comprises multiple trained image quality selector machine learning models arranged in a pipeline, and wherein a retinal fundus image is only outputted to the second stage if all of the trained image quality selector machine learning models classified the retinal fundus image as being of sufficient image quality.
7 . A machine learning method for classifying retinal features and for predicting, based on the classified retinal features, an onset or presence of Alzheimer's disease in a human subject, the method comprising:
in a processor configured to perform at least one trained machine learning model configured in a multiple-stage (multi-stage) pipeline architecture comprising at least first, second, and third stages in series, the second stage following the first stage and the third stage following the second stage, the first, second, and third stages comprising, respectively, a trained image quality selector machine learning model, a trained vessel map generator machine learning model and a trained Alzheimer disease classifier machine learning model:
receiving a retinal fundus image that has been acquired by an image acquisition system in a clinical setting;
processing the acquired retinal fundus image at a pixel level to classify one or more retinal features contained in the acquired retinal fundus image, where the trained vessel map generator machine learning model generates a respective vessel map from the retinal fundus image via vessel segmentation; and
predicting, based on the classification, whether the retinal fundus image is indicative of an onset or presence of Alzheimer's disease in the human subject, where the trained Alzheimer disease classifier machine learning model processes the respective vessel map to predict whether the retinal fundus image is indicative of the onset or presence of Alzheimer's disease, and where a treatment strategy is selected based on the prediction and communicated for implementation in the clinical setting and treatment of the human subject is initiated based upon the treatment strategy; and
wherein each of the at least first, second, and third stages are separately trained, said at least one trained machine learning model has been trained on stored retinal fundus images obtained from at least a first group of human subjects who have previously been diagnosed as having Alzheimer's disease and at least a first group of human subjects who have not previously been diagnosed as having Alzheimer's disease.
8 . The machine learning method of claim 7 , wherein the trained image quality selector machine learning model is used during training of the Alzheimer disease classifier machine learning model to classify retinal fundus images inputted to the first stage as being either of sufficient image quality or insufficient image quality based upon image composition, exposure/contrast, artifacts, and sharpness/focus and outputting retinal fundus images classified as being of sufficient image quality to the second stage, and wherein during training of the Alzheimer disease classifier machine learning model, the trained vessel map generator machine learning model generates respective vessel maps for each respective retinal fundus image inputted to the second stage and outputs the vessel maps to the third stage, and wherein during training of the Alzheimer disease classifier machine learning model, the vessel maps outputted to the third stage are used to train the Alzheimer's disease classifier model.
9 . The machine learning method of claim 7 , wherein after the Alzheimer disease classifier machine learning model has been trained to classify retinal fundus images as being from a human subject having Alzheimer's disease, a retinal fundus image obtained from a patient is processed during the processing step by the trained vessel map generator machine learning model to produce the respective vessel map that is outputted to the third stage, and wherein the respective vessel map is processed by the trained Alzheimer disease classifier machine learning model during the predicting step to predict, based on the classification, whether the retinal fundus image is indicative of an onset or presence of Alzheimer's disease in the human subject.
10 . The machine learning method of claim 7 , wherein the trained Alzheimer disease classifier machine learning model further comprises a saliency map generator for generating respective saliency maps from respective vessel maps generated by the trained vessel map generator learning machine model, where the saliency maps indicate vessel map pixels that are salient to classification to Alzheimer's disease and vessel map pixels that do not significantly contribute to the classification of Alzheimer's disease.
11 . A non-transitory computer-readable storage medium, comprising a machine learning model comprising computer instructions for execution by a processor of at least one computer system or computing device for classifying retinal features and for predicting, based on the classified retinal features, an onset or presence of Alzheimer's disease in a human subject, the machine learning model comprising:
at least one trained machine learning model, the trained machine learning model configured in a multiple-stage (multi-stage) pipeline architecture comprising at least first, second, and third stages in series, the second stage following the first stage and the third stage following the second stage, the first, second, and third stages comprising, respectively, a trained image quality selector machine learning model, a trained vessel map generator machine learning model and a trained Alzheimer disease classifier machine learning model, where execution of the at least one trained machine learning model causes the at least one computer system or computing device to at least:
receive a retinal fundus image that has been acquired by an image acquisition system in a clinical setting;
process the acquired retinal fundus image at a pixel level to classify one or more retinal features contained in the acquired retinal fundus image, where the trained vessel map generator machine learning model generates a respective vessel map from the retinal fundus image via vessel segmentation; and
predict, based on the classification, whether the retinal fundus image is indicative of an onset or presence of Alzheimer's disease in the human subject, where the trained Alzheimer disease classifier machine learning model processes the respective vessel map to predict whether the retinal fundus image is indicative of the onset or presence of Alzheimer's disease, and where a treatment strategy is selected based on the prediction and communicated for implementation in the clinical setting and treatment of the human subject is initiated based upon the treatment strategy; and
wherein each of the at least first, second, and third stages are separately trained, said at least one trained machine learning model has been trained on stored retinal fundus images obtained from at least a first group of human subjects who have previously been diagnosed as having Alzheimer's disease and at least a first group of human subjects who have not previously been diagnosed as having Alzheimer's disease.
12 . The non-transitory computer-readable storage medium of claim 11 , wherein the trained image quality selector machine learning model is used during training of the Alzheimer disease classifier machine learning model to classify retinal fundus images inputted to the first stage as being either of sufficient image quality or insufficient image quality based upon image composition, exposure/contrast, artifacts, and sharpness/focus and outputting retinal fundus images classified as being of sufficient image quality to the second stage, and wherein during training of the Alzheimer disease classifier machine learning model, the trained vessel map generator machine learning model generates respective vessel maps for each respective retinal fundus image inputted to the second stage and outputs the vessel maps to the third stage, and wherein during training of the Alzheimer disease classifier machine learning model, the vessel maps outputted to the third stage are used to train the Alzheimer's disease classifier model.
13 . The non-transitory computer-readable storage medium of claim 12 , wherein after the Alzheimer disease classifier machine learning model has been trained to classify retinal fundus images as being from a human subject having Alzheimer's disease, a retinal fundus image obtained from a patient is processed by the trained vessel map generator machine learning model to produce the respective vessel map that is outputted to the third stage, and wherein the respective vessel map is processed by the trained Alzheimer disease classifier machine learning model to predict, based on the classification, whether the retinal fundus image is indicative of an onset or presence of Alzheimer's disease in the human subject.
14 . The non-transitory computer-readable storage medium of claim 11 , wherein the Alzheimer disease classifier machine learning model comprises a saliency map generator in the third stage for generating respective saliency maps from respective vessel maps generated by the trained vessel map generator learning machine model, where the saliency maps indicate vessel map pixels that are salient to classification to Alzheimer's disease and vessel map pixels that do not significantly contribute to the classification of Alzheimer's disease.