SYSTEMS AND METHODS TO PROCESS ELECTRONIC IMAGES TO IDENTIFY ABNORMAL MORPHOLOGIES

1 . A method for identifying morphologies present in digital whole slide images, the method comprising:

receiving one or more digital whole slide images associated with a patient;

determining a plurality of foreground tiles within the one or more digital whole slide images associated with a patient;

determining, using a trained machine learning model, whether each foreground tile of the plurality of foreground tiles contains a known morphology or an unknown morphology;

upon determining that one or more foreground tiles contains an unknown morphology, providing the one or more foreground tiles with an unknown morphology to a clustering algorithm, the clustering algorithm associating each of the one or more tiles with an unknown morphology cluster; and

based on the associated unknown morphology cluster, predicting at least one outcome for the patient.

2 . The method of claim 1 , wherein the one or more foreground tiles are determined by thresholding based on pixel variance, thresholding based on minimizing intra-class intensity variance, thresholding based on maximizing inter-class intensity variance, and/or comparing foreground tile pixel values to a reference foreground distribution.

3 . The method of claim 1 , wherein determining a plurality of foreground tiles within the one or more digital whole slide images associated with a patient further comprises normalizing the digital whole slide images for magnification levels.

4 . The method of claim 1 , wherein whether each foreground tile of the plurality of foreground tiles contains a known morphology or an unknown morphology is determined using an open-set classifier.

5 . The method of claim 1 , wherein the clustering algorithm uses a Mixture Model, a K-Means Model, agglomerative clustering, and/or an Expectation-Maximization Algorithm approach.

6 . The method of claim 1 , wherein providing the one or more foreground tiles to the clustering algorithm comprises:

determining a vector of features for each foreground tile with an unknown morphology, the clustering algorithm clustering a plurality of vectors associated with foreground tiles of unknown tissue morphology.

7 . The method of claim 6 , wherein the clustering algorithm may extract the plurality of vectors using hand-engineered features, pre-trained convolutional neural network (CNN) embeddings using supervised learning, pre-trained CNN embeddings using self-supervised learning techniques, or pre-trained transformer neural network features.

8 . The method of claim 1 , wherein predicting at least one outcome for the patient further comprises:

receiving patient data associated with the unknown morphology cluster; and

determining outcome data using the received patient data.

9 . The method of claim 1 , wherein the at least one outcome comprises at least one of a patient prognosis, a patient prognosis including years of survival, likelihood of response to medication, likelihood of recurrence, likelihood of metastasis, survival rate, effective medication type, effective treatment type, and a 5-year survival rate.

10 . The method of claim 1 , wherein the at least one outcome is predicted using a binary model based on presence of unknown tiles.

11 . The method of claim 1 , further comprising visualizing the one or more foreground tiles assigned to clusters to be analyzed by a medical professional.

12 . The method of claim 1 , further comprising correlating patient outcomes with the one or more clusters to predict prognosis based on the one or more clusters.

13 . A system for identifying morphologies present in digital medical images, the system comprising:

at least one memory storing instructions; and

at least one processor configured to execute the instructions to perform operations comprising:

receive one or more digital whole slide images associated with a patient;

determine a plurality of foreground tiles within the one or more digital whole slide images associated with a patient;

determine, using a machine learning model, whether each foreground tile of the plurality of foreground tiles contains a known morphology or an unknown morphology;

upon determining that one or more foreground tiles contains an unknown morphology, provide the one or more foreground tiles with an unknown morphology to a clustering algorithm, the clustering algorithm associating each of the one or more tiles with an unknown morphology cluster; and

based on the associated unknown morphology cluster, predict at least one outcome for the patient.

14 . The system of claim 13 , wherein the one or more foreground tiles are determined by thresholding based on pixel variance, thresholding based on minimizing intra-class intensity variance, thresholding based on maximizing inter-class intensity variance, and/or comparing foreground tile pixel values to a reference foreground distribution.

15 . The system of claim 13 , wherein determining a plurality of foreground tiles within the one or more digital whole slide images associated with a patient further comprises normalizing the digital whole slide images for magnification levels.

16 . The system of claim 13 , wherein providing the one or more foreground tiles to the clustering algorithm comprises:

determining a vector of features for each tile with an unknown morphology, the clustering algorithm clustering a plurality of vectors associated with foreground tiles of unknown tissue morphology.

17 . A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations for identifying morphologies present in digital medical images, the operations comprising:

receiving one or more digital whole slide images associated with a patient;

determining a plurality of foreground tiles within the one or more digital whole slide images associated with a patient;

determining, using a machine learning model, whether each foreground tile of the plurality of foreground tiles contains a known morphology or an unknown morphology;

upon determining that one or more foreground tiles contains an unknown morphology, providing the one or more foreground tiles with an unknown morphology to a clustering algorithm, the clustering algorithm associating each of the one or more tiles with an unknown morphology cluster; and

based on the associated unknown morphology cluster, predicting at least one outcome for the patient.

18 . The non-transitory computer-readable medium of claim 17 , wherein the one or more foreground tiles are determined by thresholding based on pixel variance, thresholding based on minimizing intra-class intensity variance, thresholding based on maximizing inter-class intensity variance, and/or comparing foreground tile pixel values to a reference foreground distribution.

19 . The non-transitory computer-readable medium of claim 17 , wherein determining a plurality of foreground tiles within the one or more digital whole slide images associated with a patient further comprises normalizing the digital whole slide images for magnification levels.

20 . The non-transitory computer-readable medium of claim 17 , wherein providing the one or more foreground tiles to the clustering algorithm comprises:

determining a vector of features for each tile with an unknown morphology, the clustering algorithm clustering a plurality of vectors associated with foreground tiles of unknown tissue morphology.