Predicting response to anti-vascular endothelial growth factor therapy with computer-extracted morphology and spatial arrangement features of leakage patterns on baseline fluorescein angiography in diabetic macular edema
Embodiments facilitate prediction of anti-vascular endothelial growth (anti-VEGF) therapy response in DME patients. A first set of embodiments discussed herein relates to training of a machine learning classifier to determine a prediction for response to anti-VEGF therapy based on a set of graph-network features and a set of morphological features generated based on FA images of tissue demonstrating DME. A second set of embodiments discussed herein relates to determination of a prediction of response to anti-VEGF therapy for a DME patient (e.g., non-rebounder vs. rebounder, response vs. non-response) based on a set of graph-network features and a set of morphological features generated based on FA imagery of the patient.
1. A non-transitory computer-readable storage device storing computer-executable instructions that when executed cause a processor to perform operations, the operations comprising:
accessing a two-dimensional (2D) fluorescein angiography (FA) image of a region of interest (ROI) demonstrating diabetic macular edema (DME), where the image is associated with a patient, where the image has a plurality of pixels, a pixel having an intensity;
generating a leakage localization mask based on the FA image, where the leakage localization mask includes a plurality of leakage patches;
generating a set of leakage graphs based on the leakage localization mask, where a node of a member of the set of leakage graphs is a member of the plurality of leakage patches;
generating a set of graph-network features based on the set of leakage graphs;
extracting a set of morphological features from the FA image based on the leakage localization mask;
providing the set of graph-network features and the set of morphological features to a machine learning classifier configured to distinguish non-rebounders from rebounders in DME based on the set of graph-network features and the set of morphological features;
receiving, from the machine learning classifier, a probability that the patient is a non-rebounder, where the machine learning classifier computes the probability based on the set of graph-network features and the set of morphological features;
generating a classification of the patient as a non-rebounder or rebounder based, at least in part, on the probability; and
displaying the classification.
2. The non-transitory computer-readable storage device of claim 1 , where the FA image is a pre-anti-vascular endothelial growth factor (VEGF) treatment ultra-wide field FA (UWFA) image.
3. The non-transitory computer-readable storage device of claim 1 , where generating the leakage localization mask comprises:
accessing an early phase FA image and a late phase FA image associated with the patient;
generating a registered early phase FA image and a registered late phase FA image by registering the early phase FA image with the late phase FA image;
generating a flattened late phase image based on the registered late phase image;
generating a spectrally enhanced late phase image based on the flattened late phase image;
removing accentuated vessels from the spectrally enhanced late phase image; and
segmenting a leakage area based on the spectrally enhanced late phase image using a fixed threshold.
4. The non-transitory computer-readable storage device of claim 3 , where the early phase FA image is registered with the late phase FA image using Fourier correlation of retinal vascular patterns.
5. The non-transitory computer-readable storage device of claim 4 , where the set of graph-network features includes an edge length disorder of MST feature.
6. The non-transitory computer-readable storage device of claim 3 , where generating the flattened late phase image comprises removing an intensity gradient of the optic disc to an image periphery.
7. The non-transitory computer-readable storage device of claim 6 , where generating the spectrally enhanced late phase image comprises equalizing leakage regions represented in the flattened late phase.
8. The non-transitory computer-readable storage device of claim 3 , where removing accentuated vessels from the spectrally enhanced late phase image comprises:
filtering vessels in the registered early phase FA image using a Gaussian convolution kernel;
spectrally enhancing the filtered vessels; and
subtracting the spectrally enhanced filtered vessels from the registered early phase FA image.
9. The non-transitory computer-readable storage device of claim 1 , where the set of leakage graphs includes a minimum spanning tree (MST) graph, where a node of the MST graph is a member of the plurality of leakage patches.
10. The non-transitory computer-readable storage device of claim 1 , where the set of morphological features includes at least one of an area feature, a distance to N nearest neighbors feature, or a disorder of variance of distance to N nearest neighbors feature.
11. The non-transitory computer-readable storage device of claim 1 , where the machine learning classifier is quadratic discriminant analysis (QDA) classifier.
12. The non-transitory computer-readable storage device of claim 11 , the operations further comprising training the machine learning classifier and optionally testing the machine learning classifier.
13. The non-transitory computer-readable storage device of claim 1 , the operations further comprising generating a personalized DME treatment plan based, at least in part, on the classification, and optionally displaying the personalized DME treatment plan.
14. An apparatus comprising:
a processor;
a memory configured to store a two-dimensional (2D) fluorescein angiography (FA) image of a region of interest (ROI) demonstrating diabetic macular edema (DME), where the image is associated with a patient, where the image has a plurality of pixels, a pixel having an intensity;
an input/output (I/O) interface;
a set of circuits; and
an interface that connects the processor, the memory, the I/O interface, and the set of circuits, the set of circuits comprising:
an image acquisition circuit configured to access a 2D FA image of an ROI demonstrating DME, where the image is associated with a patient, where the image has a plurality of pixels, a pixel having an intensity;
a leakage localization circuit configured to:
generate a leakage localization mask based on the 2D FA image, where the leakage localization mask includes a plurality of leakage patches;
a graph-network generation circuit configured to:
generate a set of leakage graphs based on the leakage localization mask, where a node of a member of the set of leakage graphs is a member of the plurality of leakage patches; and
generate a set of graph-network features based on the set of leakage graphs;
a morphology circuit configured to:
extract a set of morphological features from the FA image based on the leakage localization mask;
a DME classification circuit configured to:
compute a probability that the patient is a non-rebounder based on the set of graph-network features and the set of morphological features; and
generate a classification of the patient as a non-rebounder or rebounder based, at least in part, on the probability; and
a display circuit configured to displaying the classification and to optionally display at least one of the probability, the set of graph-network features, the set of morphological features, a member of set of leakage graphs, or the image.
15. The apparatus of claim 14 , where the set of leakage graphs includes a minimum spanning tree (MST) graph, where a node of the MST graph is a member of the plurality of leakage patches, and where the set of graph-network features includes an edge length disorder of MST feature.
16. The apparatus of claim 14 , where the set of morphological features includes at least one of an area feature, a distance to N nearest neighbors feature, or a disorder of variance of distance to N nearest neighbors feature.
17. The apparatus of claim 14 , where the 2D FA image is a pre-anti-vascular endothelial growth factor (VEGF) treatment ultra-wide field FA (UWFA) image.
18. The apparatus of claim 14 , the set of circuits further comprising a training and testing circuit configured to:
train the machine learning classifier; and
optionally test the machine learning classifier.
19. The apparatus of claim 14 , the set of circuits further comprising a DME personalized treatment plan circuit configured to:
generate a personalized DME treatment plan based, at least in part, on the classification; and
optionally control the display circuit to display the personalized DME treatment plan.
20. A non-transitory computer-readable storage device storing computer-executable instructions that when executed by a computer control the computer to perform a method, the method comprising:
accessing a pre-anti-vascular endothelial growth factor (anti-VEGF) two-dimensional (2D) ultra-wide field fluorescein angiography (UWFA) image of a region of interest (ROI) demonstrating diabetic macular edema (DME), where the image is associated with a patient, the image having a plurality of pixels, a pixel having an intensity;
generating a leakage localization mask based on the UWFA image, where the leakage localization mask includes a plurality of leakage patches;
generating a set of leakage graphs based on the leakage localization mask, where the set of leakage graphs includes a minimum spanning tree (MST) graph, where a node of the MST graph is a member of the plurality of leakage patches;
generating a set of graph-network features based on the set of leakage graphs, where the set of graph-network features includes an edge length disorder of MST feature;
extracting a set of morphological features from the UWFA image based on the leakage localization mask, where the set of morphological features includes at least one of an area feature, a distance to N nearest neighbors feature, or a disorder of variance of distance to N nearest neighbors feature;
providing the set of graph-network features and the set of morphological features to a quadratic discriminant analysis (QDA) machine learning classifier configured to distinguish non-rebounders from rebounders in DME based on the set of graph-network features and the set of morphological features;
receiving, from the machine learning classifier, a probability that the patient is a non-rebounder, where the machine learning classifier computes the probability based on the set of graph-network features and the set of morphological features;
generating a classification of the patient as a non-rebounder or rebounder based, at least in part, on the probability;
generating a personalized DME treatment plan based, at least in part, on the classification; and
displaying the classification and optionally displaying at least one of the probability, the set of graph-network features, the set of morphological features, a member of the set of leakage graphs, or the UWFA image.