Systems and methods to process electronic images to identify tumor subclones and relationships among subclones
A computer-implemented method for detecting tumor subclones may include receiving one or more digital images into a digital storage device, the one or more digital images including images of a tumor of a patient, detecting one or more neoplasms in the one or more received digital images for each patient, extracting one or more visual features from each detected neoplasm, determining a hierarchy dendrogram based on the detected one or more neoplasms and the extracted one or more visual features for each detected neoplasm, determining one or more leaf nodes based on the determined hierarchy dendrogram, and determining whether there are two or more neoplasms among the detected one or more neoplasms that originated independently.
1. A computer-implemented method for detecting tumor subclones, the method comprising: receiving one or more digital images into a digital storage device, the one or more digital images including images of a tumor of a patient; detecting one or more neoplasms in the one or more received digital images for each patient; extracting one or more visual features from each detected neoplasm; determining a hierarchy dendrogram based on the detected one or more neoplasms and the extracted one or more visual features for each of the detected one or more neoplasms; determining one or more leaf nodes based on the determined hierarchy dendrogram; determining, based on the determined hierarchy dendrogram, whether there are two or more neoplasms among the detected one or more neoplasms that originated independently; and displaying and/or storing the determined two or more neoplasms.
2. The method of claim 1 , further comprising determining a spatial location for each neoplasm or a determined neoplasm region, wherein determining the hierarchy dendrogram is further based on the determined spatial location.
3. The method of claim 1 , further comprising using hierarchical clustering on the extracted one or more visual features, wherein determining the hierarchy dendrogram is based on the hierarchical clustering.
4. The method of claim 1 , further comprising determining a similarity of each extracted visual feature to each other extracted visual feature.
5. The method of claim 4 , wherein determining the similarity includes applying a trained similarity metric system on all pair-wise combinations of the one or more extracted visual features.
6. The method of claim 1 , further comprising determining a score for each detected neoplasm, wherein determining the hierarchy dendrogram is further based on the determined score for each detected neoplasm.
7. The method of claim 6 , wherein determining the score includes applying a trained scoring system.
8. The method of claim 6 , wherein the score corresponds to an outcome or treatment response.
9. The method of claim 6 , further comprising using hierarchical clustering on the determined score for each detected neoplasm, wherein determining the hierarchy dendrogram is further based on the hierarchical clustering.
10. The method of claim 1 , further comprising dividing each received digital image into sub-regions, and determining which sub-regions among the divided sub-regions include a neoplasm.
11. The method of claim 1 , further comprising determining a type for each detected neoplasm.
12. The method of claim 1 , wherein detecting one or more neoplasms in the one or more received digital images for each patient is performed using a trained machine learning model.
13. The method of claim 1 , further comprising outputting the hierarchy dendrogram on a display.
14. The method of claim 13 , further comprising determining that two or more neoplasms among the detected one or more neoplasms originated independently, wherein the output hierarchy dendrogram is color coded based on the determined two or more neoplasms that originated independently.
15. A system for detecting tumor subclones, the system comprising: a data storage device storing instructions for detecting tumor subclones in an electronic storage medium; and a processor configured to execute the instructions to perform a method including: receiving one or more digital images into a digital storage device, the one or more digital images including images of a tumor of a patient; detecting one or more neoplasms in the one or more received digital images for each patient; extracting one or more visual features from each detected neoplasm; determining a hierarchy dendrogram based on the detected one or more neoplasms and the extracted one or more visual features for each of the detected one or more neoplasms; determining one or more leaf nodes based on the determined hierarchy dendrogram; determining, based on the determined hierarchy dendrogram, whether there are two or more neoplasms among the detected one or more neoplasms that originated independently; and displaying and/or storing the determined two or more neoplasms.
16. The system of claim 15 , wherein the system is further configured for:
using hierarchical clustering on the extracted one or more visual features, wherein determining the hierarchy dendrogram is based on the hierarchical clustering.
17. The system of claim 15 , wherein detecting one or more neoplasms in the one or more received digital images for each patient is performed using a trained machine learning model.
18. A non-transitory machine-readable medium storing instructions that, when executed by a computing system, causes the computing system to perform a method for detecting tumor subclones, the method including: receiving one or more digital images into a digital storage device, the one or more digital images including images of a tumor of a patient; detecting one or more neoplasms in the one or more received digital images for each patient; extracting one or more visual features from each of the detected one or more neoplasms; determining a hierarchy dendrogram based on the detected one or more neoplasms and the extracted one or more visual features for each detected neoplasm; determining one or more leaf nodes based on the determined hierarchy dendrogram; determining, based on the determined hierarchy dendrogram, whether there are two or more neoplasms among the detected one or more neoplasms that originated independently; and displaying and/or storing the determined two or more neoplasms.
19. The non-transitory machine-readable medium of claim 18 , the method further comprising:
using hierarchical clustering on the extracted one or more visual features, wherein determining the hierarchy dendrogram is based on the hierarchical clustering.
20. The non-transitory machine-readable medium of claim 18 , wherein detecting one or more neoplasms in the one or more received digital images for each patient is performed using a trained machine learning model.