AUTONOMOUS DIAGNOSIS OF A DISORDER IN A PATIENT FROM IMAGE ANALYSIS
Provide are systems methods and devices for diagnosing disease in medical images. In certain aspects, disclosed is a method for training a neural network to detect features in a retinal image including the steps of: a) extracting one or more features images from a Train_0 set, a Test_0 set, a Train_1 set and a Test_1 set; b) combining and randomizing the feature images from Train_0 and Train_1 into a Training data set; c) combining and randomizing the feature images from Test_0 and Test_1 into a testing dataset; d) training a plurality of neural networks having different architectures using a subset of the training dataset while testing on a subset of the testing dataset; e) identifying the best neural network based on each of the plurality of neural networks performance on the testing data set; f) inputting images from Test_0, Train_1, Train_0 and Test_1 to the best neural network and identifying a limited number of false positives and false negative and adding the false positives and false negatives to the training dataset and testing dataset; and g) repeating steps d)-g) until an objective performance threshold is reached.
1 . A method for training a diagnostic model for diagnosing a disease condition in a patient, the method comprising:
accessing a plurality of input images, each input image including a portion of body of a patient selected from a plurality of patients;
accessing a label for each input image that indicates whether the selected patient in the input image has a disease condition;
obtaining a plurality of training examples, each training example corresponding to a given input image and comprising:
for each respective location in the given input image, an indication that the given input image contains an object of interest at the respective location, wherein the object of interest is indicative of a disease, and
the label of the given input image; and
for a diagnostic model, the diagnostic model comprising a machine learning model that is configured to output a diagnosis of a disease condition based on an input of indications of whether there is an object of interest at each location within a sample image:
training the diagnostic model by repeatedly applying a training example from the plurality of training examples to the diagnostic model and updating parameters of the diagnostic model to improve an objective performance threshold thereof, and
stopping the training after the objective performance threshold satisfies a condition.
2 . The method of claim 1 , wherein the indication that the given input image contains an object of interest for each of one or more locations comprises a mathematical model of the retinal object.
3 . The method of claim 1 , wherein the input of indications of whether there is an object of interest at each location within a sample image comprises a heat map indicating the likelihood that the sample image contains an object of interest for each location in the sample image.
4 . The method of claim 1 , wherein the input of indications of whether there is an object of interest at each location within a sample image comprises a point-wise output corresponding to indications that the sample image contains an object of interest at each location in the sample image.
5 . The method of claim 1 , wherein one or more of the objects of interests is indicative of disease.
6 . The method of claim 1 , wherein the portion of the patient's body includes at least a portion of the patient's eye, and the determined diagnosis of a disease condition in the patient comprises a diagnosis of a disorder manifesting in the retina.
7 . The method of claim 6 , wherein one or more of the object of interests is selected from a group consisting of: a microaneurysm, a dot hemorrhage, a flame-shaped hemorrhage, a sub-intimal hemorrhage, a sub-retinal hemorrhage, a pre-retinal hemorrhage, a micro-infarction, a cotton-wool spot, and a yellow exudate.
8 . The method of claim 1 , wherein the input image is obtained by at least one of: computed tomography (CT), magnetic resonance imaging (MM), computed radiography, magnetic resonance, angioscopy, optical coherence tomography, color flow Doppler, cystoscopy, diaphanography, echocardiography, fluorescein angiography, laparoscopy, magnetic resonance angiography, positron emission tomography, single-photon emission computed tomography, x-ray angiography, nuclear medicine, biomagnetic imaging, colposcopy, duplex Doppler, digital microscopy, endoscopy, fundoscopy, laser surface scanning, magnetic resonance spectroscopy, radiographic imaging, thermography, and radio fluoroscopy.
9 . A diagnostic product for diagnosing a disease condition in a patient, wherein the diagnostic product is stored on a non-transitory computer readable medium and is manufactured by a process comprising:
accessing a plurality of input images, each input image including a portion of body of a patient selected from a plurality of patients;
accessing a label for each input image that indicates whether the selected patient in the input image has a disease condition;
obtaining a plurality of training examples, each training example corresponding to a given input image and comprising:
for each respective location in the given input image, an indication that the given input image contains an object of interest at the respective location, wherein the object of interest is indicative of a disease, and
the label of the given input image;
for a diagnostic model, the diagnostic model comprising a machine learning model that is configured to output a diagnosis of a disease condition based on an input of indications of whether there is an object of interest at each location within a sample image:
training the diagnostic model by repeatedly applying a training example from the plurality of training examples to the diagnostic model and updating parameters of the diagnostic model to improve an objective performance threshold thereof, and
stopping the training after the objective performance threshold satisfies a condition; and
storing the updated parameters for the diagnostic model on the computer readable storage medium.
10 . The diagnostic product of claim 9 , wherein the indication that the given input image contains an object of interest for each of one or more locations comprises a mathematical model of the retinal object.
11 . The diagnostic product of claim 9 , wherein the input of indications of whether there is an object of interest at each location within a sample image comprises a heat map indicating the likelihood that the sample image contains an object of interest location in the sample image.
12 . The diagnostic product of claim 9 , wherein the input of indications of whether there is an object of interest at each location within a sample image comprises a point-wise output corresponding to indications that the sample image contains an object of interest at each location in the sample image.
13 . The diagnostic product of claim 9 , wherein one or more of the objects of interests is indicative of disease.
14 . The diagnostic product of claim 9 , wherein the portion of the patient's body includes at least a portion of the patient's eye, and the determined diagnosis of a disease condition in the patient comprises a diagnosis of a disorder manifesting in the retina.
15 . The diagnostic product of claim 14 , wherein one or more of the object of interests is selected from a group consisting of: a microaneurysm, a dot hemorrhage, a flame-shaped hemorrhage, a sub-intimal hemorrhage, a sub-retinal hemorrhage, a pre-retinal hemorrhage, a micro-infarction, a cotton-wool spot, and a yellow exudate.
16 . The diagnostic product of claim 9 , wherein the input image is obtained by at least one of: computed tomography (CT), magnetic resonance imaging (MRI), computed radiography, magnetic resonance, angioscopy, optical coherence tomography, color flow Doppler, cystoscopy, diaphanography, echocardiography, fluorescein angiography, laparoscopy, magnetic resonance angiography, positron emission tomography, single-photon emission computed tomography, x-ray angiography, nuclear medicine, biomagnetic imaging, colposcopy, duplex Doppler, digital microscopy, endoscopy, fundoscopy, laser surface scanning, magnetic resonance spectroscopy, radiographic imaging, thermography, and radio fluoroscopy.
17 . A method for diagnosing a disorder manifesting in the retina, the method comprising:
receiving a retinal image of at least a portion of a patient's eye;
obtaining a set of samples of the retinal image, each sample corresponding to a location in the retinal image;
for each of the set of samples, applying the sample to a trained feature detection model, the feature detection model comprising a multilevel neural network that is configured to output a likelihood that the sample contains a retinal image object;
determining a spatial feature map that comprises, for each of one or more of the samples, the likelihood from the multilevel neural network that the sample contains a retinal image object and the location in the retinal image of the retinal image object;
applying the spatial feature map to a diagnostic model, the diagnostic model comprising a trained machine learning model that is configured to output a diagnosis of a disorder manifesting in the retina based on an input spatial feature map; and
outputting the determined diagnosis of a disorder manifesting in the retina obtained from the diagnostic model.
18 . The method of claim 17 , wherein applying the input image to the feature extraction model comprises:
obtaining a set of samples of the input image, each sample corresponding to a location in the input image; and
for each sample of the set of samples, applying the sample to the feature extraction model, the feature extraction model configured to output an indication that the sample contains an object of interest.
19 . The method of claim 17 , wherein the likelihood that the sample contains a retinal image object comprises a mathematical model of the retinal object.
20 . The method of claim 17 , wherein the spatial feature map comprises a heat map indicating likelihoods that the sample contains a retinal image object in the retinal image.