Autonomous diagnosis of a disorder in a patient from image analysis
A device receives an input image of a portion of a patient's body, and applies the input image to a feature extraction model, the feature extraction model comprising a trained machine learning model that is configured to generate an output that comprises, for each respective location of a plurality of locations in the input image, an indication that the input image contains an object of interest that is indicative of a presence of a disease state at the respective location. The device applies the output of the feature extraction model to a diagnostic model, the diagnostic model comprising a trained machine learning model that is configured to output a diagnosis of a disease condition in the patient based on the output of the feature extraction model. The device outputs the determined diagnosis of a disease condition in the patient obtained from the diagnostic model.
1. A method for diagnosing a disease condition in a patient, the method comprising:
receiving an input image of a portion of a patient's body;
applying the input image to a feature extraction model, the input image comprising a plurality of locations, the feature extraction model comprising a first trained machine learning model that is configured to generate an output that comprises, for each respective location of each of the plurality of locations in the input image, an indication that the input image contains an object of interest at the respective location, where the object of interest is indicative of a disease;
applying the output of the feature extraction model to a diagnostic model, the diagnostic model comprising a second trained machine learning model that is configured to output a diagnosis of a disease condition in the patient based on taking as input the indications for each of the plurality of locations in the input image as output by the feature extraction model, where the second trained machine learning model is trained by:
applying ground truth training data that comprises labels about whether a corresponding image depicts a disease and, at each location of a plurality of locations of the corresponding image, indicia that the corresponding image contains an object of interest that is indicative of a disease, to the second machine learning model, and
adjusting the second machine learning model to fit the ground truth training data, thereby causing the second machine learning model to become trained; and
outputting the determined diagnosis of the disease condition in the patient obtained from the diagnostic model.
2. The method of claim 1 , wherein applying the input image to the feature extraction model comprises:
obtaining a set of samples of the input image, each sample corresponding to a location in the input image; and
for each sample of the set of samples, applying the sample to the feature extraction model, the feature extraction model configured to output an indication that the sample contains an object of interest.
3. The method of claim 2 , wherein the set of samples is obtained by at least one of random sampling and grid sampling.
4. The method of claim 1 , wherein the indication that the input image contains an object of interest at one or more locations comprises a mathematical representation of the object of interest.
5. The method of claim 1 , wherein the output of the feature extraction model comprises a heat map corresponding to indications that the input image contains an object of interest at the plurality of locations in the input image.
6. The method of claim 1 , wherein the output of the feature extraction model comprises a point-wise output corresponding to indications that the input image contains an object of interest at the plurality of locations in the input image.
7. The method of claim 1 , wherein one or more of the objects of interest is indicative of disease.
8. The method of claim 1 , wherein the portion of the patient's body includes at least a portion of the patient's eye, and the determined diagnosis of the disease condition in the patient comprises a diagnosis of a disorder manifesting in the retina.
9. The method of claim 8 , wherein one or more of the objects of interest is selected from a group consisting of: a microaneurysm, a dot hemorrhage, a flame-shaped hemorrhage, a sub-intimal hemorrhage, a sub-retinal hemorrhage, a pre-retinal hemorrhage, a micro-infarction, a cotton-wool spot, and a yellow exudate.
10. The method of claim 1 , wherein the input image is obtained by at least one of: computed tomography (CT), magnetic resonance imaging (MRI), computed radiography, magnetic resonance, angioscopy, optical coherence tomography, color flow Doppler, cystoscopy, diaphanography, echocardiography, fluorescein angiography, laparoscopy, magnetic resonance angiography, positron emission tomography, single-photon emission computed tomography, x-ray angiography, nuclear medicine, biomagnetic imaging, colposcopy, duplex Doppler, digital microscopy, endoscopy, fundoscopy, laser surface scanning, magnetic resonance spectroscopy, radiographic imaging, thermography, and radio fluoroscopy.
11. A non-transitory computer-readable medium comprising memory with instructions encoded thereon for diagnosing a disease condition in a patient, the instructions, when executed, causing one or more processors to perform operations, the instructions comprising instructions to:
receive an input image of a portion of a patient's body;
apply the input image to a feature extraction model, the input image comprising a plurality of locations, the feature extraction model comprising a first trained machine learning model that is configured to generate an output that comprises, for each respective location of each of the plurality of locations in the input image, an indication that the input image contains an object of interest at the respective location, where the object of interest is indicative of a disease;
apply the output of the feature extraction model to a diagnostic model, the diagnostic model comprising a second trained machine learning model that is configured to output a diagnosis of a disease condition in the patient based on taking as input the indications for each of the plurality of locations in the input image as output by the feature extraction model, where the second trained machine learning model is trained by:
applying ground truth training data that comprises labels about whether a corresponding image depicts a disease and, at each location of a plurality of locations of the corresponding image, indicia that the corresponding image contains an object of interest that is indicative of a disease, to the second machine learning model, and
adjusting the second machine learning model to fit the ground truth training data, thereby causing the second machine learning model to become trained; and
output the determined diagnosis of the disease condition in the patient obtained from the diagnostic model.
12. The non-transitory computer-readable medium of claim 11 , wherein the instructions to apply the input image to the feature extraction model comprise instructions to:
obtain a set of samples of the input image, each sample corresponding to a location in the input image; and
for each sample of the set of samples, apply the sample to the feature extraction model, the feature extraction model configured to output an indication that the sample contains an object of interest.
13. The non-transitory computer-readable medium of claim 12 , wherein the set of samples is obtained by at least one of random sampling and grid sampling.
14. The non-transitory computer-readable medium of claim 11 , wherein the indication that the input image contains an object of interest at one or more locations comprises a mathematical representation of the object of interest.
15. The non-transitory computer-readable medium of claim 11 , wherein the output of the feature extraction model comprises a heat map corresponding to indications that the input image contains an object of interest at the plurality of locations in the input image.
16. The non-transitory computer-readable medium of claim 11 , wherein the output of the feature extraction model comprises a point-wise output corresponding to indications that the input image contains an object of interest at the plurality of locations in the input image.
17. The non-transitory computer-readable medium of claim 11 , wherein one or more of the objects of interest is indicative of disease.
18. The non-transitory computer-readable medium of claim 11 , wherein the portion of the patient's body includes at least a portion of the patient's eye, and the determined diagnosis of the disease condition in the patient comprises a diagnosis of a disorder manifesting in the retina.
19. The non-transitory computer-readable medium of claim 18 , wherein one or more of the objects of interest is selected from a group consisting of: a microaneurysm, a dot hemorrhage, a flame-shaped hemorrhage, a sub-intimal hemorrhage, a sub-retinal hemorrhage, a pre-retinal hemorrhage, a micro-infarction, a cotton-wool spot, and a yellow exudate.
20. The non-transitory computer-readable medium of claim 11 , wherein the input image is obtained by at least one of: computed tomography (CT), magnetic resonance imaging (MRI), computed radiography, magnetic resonance, angioscopy, optical coherence tomography, color flow Doppler, cystoscopy, diaphanography, echocardiography, fluorescein angiography, laparoscopy, magnetic resonance angiography, positron emission tomography, single-photon emission computed tomography, x-ray angiography, nuclear medicine, biomagnetic imaging, colposcopy, duplex Doppler, digital microscopy, endoscopy, fundoscopy, laser surface scanning, magnetic resonance spectroscopy, radiographic imaging, thermography, and radio fluoroscopy.
21. A system for diagnosing a disease condition in a patient, the system comprising:
memory with instructions encoded thereon; and
one or more processors that, when executing the instructions, are caused to perform operations comprising:
receiving an input image of a portion of a patient's body;
applying the input image to a feature extraction model, the input image comprising a plurality of locations, the feature extraction model comprising a trained machine learning model that is configured to generate an output that comprises, for each respective location of each of the plurality of locations in the input image, an indication that the input image contains an object of interest at the respective location, where the object of interest is indicative of a disease;
applying the output of the feature extraction model to a diagnostic model, the diagnostic model comprising a second trained machine learning model that is configured to output a diagnosis of a disease condition in the patient based on taking as input the indications for each of the plurality of locations in the input image as output by the feature extraction model, where the second trained machine learning model is trained by:
applying ground truth training data that comprises labels about whether a corresponding image depicts a disease and, at each location of a plurality of locations of the corresponding image, indicia that the corresponding image contains an object of interest that is indicative of a disease, to the second machine learning model, and
adjusting the second machine learning model to fit the ground truth training data, thereby causing the second machine learning model to become trained; and
outputting the determined diagnosis of the disease condition in the patient obtained from the diagnostic model.
22. The system of claim 21 , wherein applying the input image to the feature extraction model comprises:
obtaining a set of samples of the input image, each sample corresponding to a location in the input image; and
for each sample of the set of samples, applying the sample to the feature extraction model, the feature detection model configured to output an indication that the sample contains an object of interest.
23. The system of claim 22 , wherein the set of samples is obtained by at least one of random sampling and grid sampling.
24. The system of claim 21 , wherein the indication that the input image contains an object of interest at one or more locations comprises a mathematical representation of the object of interest.
25. The system of claim 21 , wherein the output of the feature extraction model comprises a heat map corresponding to indications that the input image contains an object of interest at the plurality of locations in the input image.
26. The system of claim 21 , wherein the output of the feature extraction model comprises a point-wise output corresponding to indications that the input image contains an object of interest at the plurality of locations in the input image.
27. The system of claim 21 , wherein one or more of the objects of interest is indicative of disease.
28. The system of claim 21 , wherein the portion of the patient's body includes at least a portion of the patient's eye, and the determined diagnosis of the disease condition in the patient comprises a diagnosis of a disorder manifesting in the retina.
29. The system of claim 28 , wherein one or more of the objects of interest is selected from a group consisting of: a microaneurysm, a dot hemorrhage, a flame-shaped hemorrhage, a sub-intimal hemorrhage, a sub-retinal hemorrhage, a pre-retinal hemorrhage, a micro-infarction, a cotton-wool spot, and a yellow exudate.
30. The system of claim 21 , wherein the input image is obtained by at least one of: computed tomography (CT), magnetic resonance imaging (MRI), computed radiography, magnetic resonance, angioscopy, optical coherence tomography, color flow Doppler, cystoscopy, diaphanography, echocardiography, fluorescein angiography, laparoscopy, magnetic resonance angiography, positron emission tomography, single-photon emission computed tomography, x-ray angiography, nuclear medicine, biomagnetic imaging, colposcopy, duplex Doppler, digital microscopy, endoscopy, fundoscopy, laser surface scanning, magnetic resonance spectroscopy, radiographic imaging, thermography, and radio fluoroscopy.