Assessment of endothelial cells and corneas at risk from ophthalmological images
Embodiments discussed herein facilitate determining a prognosis for keratoplasty based on segmented endothelial cells. One example embodiment comprises a computer-readable medium storing computer-executable instructions that, when executed, cause a processor to perform operations, comprising: accessing an optical microscopy image comprising a set of corneal endothelial cells of a patient of a keratoplasty; segmenting, based at least in part on a first model, a plurality of corneal endothelial cells of the set of corneal endothelial cells; calculating one or more features based on the segmented plurality of corneal endothelial cells; and generating, via a second model trained based at least on the one or more features, a prognosis associated with the keratoplasty.
1. A non-transitory computer-readable medium storing computer-executable instructions that, when executed, cause a processor to perform operations, comprising:
accessing an optical microscopy image comprising a set of corneal endothelial cells of a patient of a keratoplasty;
segmenting, based at least in part on a first model, a plurality of corneal endothelial cells of the set of corneal endothelial cells;
calculating one or more features from the segmented plurality of corneal endothelial cells, wherein calculating the one or more features includes calculating bright pigments, dark nuclei, and a thickness of dark gaps between cells of the segmented plurality of corneal endothelial cells; and
generating, via a second model using the one or more features, a prognosis associated with the keratoplasty at a time following the keratoplasty.
2. The non-transitory computer-readable medium of claim 1 , wherein the one or more features comprise statistical measures over the optical microscopy image, the statistical measures including one or more of a skewness, a kurtosis, and a binned histogram.
3. The non-transitory computer-readable medium of claim 1 , wherein the one or more features are calculated after segmenting the plurality of corneal endothelial cells.
4. The non-transitory computer-readable medium of claim 1 , wherein the one or more features comprise one or more graph features associated with the segmented plurality of corneal endothelial cells, the one or more graph features describing an arrangement between the segmented plurality of corneal endothelial cells.
5. The non-transitory computer-readable medium of claim 1 , wherein the one or more features comprise one or more of an endothelial cell density, a coefficient of variation of cell area, or a hexagonality associated with the segmented plurality of corneal endothelial cells.
6. The non-transitory computer-readable medium of claim 1 , wherein the operations further comprise performing preprocessing on the optical microscopy image.
7. The non-transitory computer-readable medium of claim 1 , wherein the segmenting comprises;
correcting illumination artifacts across an image area;
generating pixel-wise class probability maps; and
thresholding the pixel-wise class probability maps to generate a final binary segmentation map.
8. The non-transitory computer-readable medium of claim 7 , wherein the thresholding is calculated within a sliding window at each pixel location, the thresholding calculating an adaptive threshold value that is equal to:
T=m[ 1+ k (σ/σ dyn −1)]; and
wherein m is a mean intensity in a pixel neighborhood, σ is a standard deviation in the pixel neighborhood, σ dyn is a difference between maximum and minimum standard deviations across the optical microscopy image, and k is a tunable parameter.
9. The non-transitory computer-readable medium of claim 1 , wherein the operations further comprise:
receiving a segmentation result from the first model, the segmentation result having segmentation errors; and
highlighting cells within the segmentation result that are deemed to have been segmented incorrectly by the first model to a user by way of a graphical user interface (GUI), wherein the user can manually erase or edit cell boundaries corresponding to cells that have been highlighted.
10. The non-transitory computer-readable medium of claim 1 , wherein the one or more features further comprise one or more graph features include an average area extracted from a Voronoi graph, an average eccentricity extracted from a cell cluster graph, or a standard deviation of an edge length extracted from a minimum spanning tree.
11. The non-transitory computer-readable medium of claim 1 , wherein the one or more features further comprise hand-crafted features including irregular cell morphology, irregular distribution, cell nuclei, cellular reflectivity, and wall thickness.
12. The non-transitory computer-readable medium of claim 1 , wherein calculating the dark nuclei comprises:
correcting intensity background variations and reducing noise of the segmented plurality of corneal endothelial cells; and
applyinh a conditional dilation to a threshold.
13. The non-transitory computer-readable medium of claim 1 , wherein calculating the one or more features comprises:
correcting intensity background variations and reducing noise of the segmented plurality of corneal endothelial cells; and
identifying the dark nuclei and the bright pigments using the segmented plurality of corneal endothelial cells after correcting the intensity background variations and reducing the noise.
14. A non-transitory computer-readable medium storing computer-executable instructions that, when executed, cause a processor to perform operations, comprising:
accessing a training set comprising a plurality of optical microscopy images, wherein each optical microscopy image of the training set comprises an associated set of corneal endothelial cells of a patient of a keratoplasty associated with that optical microscopy image, and wherein each optical microscopy image of the training set is associated with a ground truth segmentation of a plurality of corneal endothelial cells of the set of corneal endothelial cells of that optical microscopy image;
based at least on each optical microscopy image and the ground truth segmentation for each optical microscopy image, training a deep learning model to segment the plurality of corneal endothelial cells of the set of corneal endothelial cells;
calculating one or more features from the segmented plurality of corneal endothelial cells, wherein calculating the one or more features includes calculating bright pigments, dark nuclei, and a thickness of dark gaps between cells of the segmented plurality of corneal endothelial cells; and
generating, via a second model trained based at least on the one or more features, a prognosis associated with the keratoplasty at a time following the keratoplasty.
15. The non-transitory computer-readable medium of claim 14 , wherein the deep learning model is a convolutional neural network (CNN).
16. The non-transitory computer-readable medium of claim 14 , wherein the operations further comprise performing preprocessing on each optical microscopy image of the training set.
17. A method, comprising:
accessing an optical microscopy image comprising corneal endothelial cells of a patient of a keratoplasty;
segmenting, based at least in part on a first model, a plurality of corneal endothelial cells of the optical microscopy image;
receiving a segmentation result from the first model, the segmentation result having segmentation errors; and
highlighting cells within the segmentation result that are deemed to have been segmented incorrectly by the first model to a user, wherein the user can manually erase or edit cell boundaries corresponding to cells that have been highlighted;
calculating one or more features from the segmented plurality of corneal endothelial cells; and
generating, via a second model using the one or more features, a prognosis associated with the keratoplasty at a time following the keratoplasty.
18. The method of claim 17 , wherein the segmenting includes:
correcting illumination artifacts across an image area by generating a low-pass background image using a Gaussian blur and dividing an original image with the low-pass background image;
generating pixel-wise class probability maps; and
performing thresholding and morphological processing operations on the pixel-wise class probability maps to generate a final binary segmentation map.
19. The method of claim 17 , wherein the one or more features comprise one or more graph features associated with the segmented plurality of corneal endothelial cells, the one or more graph features describing an arrangement between the segmented plurality of corneal endothelial cells.
20. The method of claim 17 , wherein the one or more features comprise one or more local or global graph features associated with the segmented plurality of corneal endothelial cells.
21. The method of claim 20 ,
wherein the one or more features further comprise endothelial cell density (ECD), coefficient of variation (CV) of cell area, and hexagonality (HEX); and
wherein the CV of cell area is determined by:
calculating an individual cell area for each of the plurality of corneal endothelial cells, the individual cell areas respectively including a cell border;
calculating a standard deviation and a mean of the individual cell areas; and
determining the CV of the individual cell area as a ratio of the standard deviation and the mean of individual cell areas.
22. The method of claim 17 , wherein calculating the one or more features includes calculating dark nuclei, bright pigments, and a thickness of dark gaps between cells of the segmented plurality of corneal endothelial cells.
23. The method of claim 22 , wherein calculating the one or more features associated with dark nuclei comprises:
correcting intensity background variations and reducing noise of the segmented plurality of corneal endothelial cells; and
applying a conditional dilation to a threshold.
24. The method of claim 17 , wherein the one or more features further comprise one or more graph features including an average area extracted from a Voronoi graph, an average eccentricity extracted from a cell cluster graph, and a standard deviation of an edge length extracted from a minimum spanning tree.