Systems and methods for predicting patient outcome to cancer therapy

1. A method of predicting patient response to therapy, comprising:

receiving a plurality of patient tissue image slides;

receiving a plurality of patient outcome data;

dividing the image slides into patches;

receiving a plurality of labels correlated with a disease or lack thereof, wherein each label comprises of a morphological type, wherein a morphological type corresponds to molecular changes associated with the disease or lack of molecular changes associated with lack of the disease;

converting each patch to a patch vector;

training an artificial intelligence network, based on the plurality of labels to identify the morphological types associated with each label in the image slides;

generating labeled patch vectors, wherein the label of a patch vector is assigned based, at least partly, on the plurality of the labels and the morphological type expressed in the patch;

clustering the labeled patch vectors, within each label, with an unsupervised artificial intelligence network, wherein the clustering groups the patch vectors within each label to one or more morphological subtypes expressed in the patch corresponding to the labeled patch vector and a patient outcome, wherein clustering comprises clustering similar patch vectors within each label, and wherein morphological subtypes comprise unknown morphological sub-patterns within a morphological type;

generating a patch-level score for each patch based at least partly on the cluster to which the patch vector of the patch belongs;

generating a patient-level score for each patient, at least partly based on the patch-level scores generated for each patient; and

predicting a patient response to a therapy, based at least in part, on the patient-level score for the patient.

2. The method of claim 1 , wherein generating the patch-level score further comprises:

converting regions surrounding each patch to microenvironment vectors;

obtaining mean vectors by averaging the patch vectors corresponding to the patches with the microenvironment vectors;

clustering the mean vectors;

sampling input mean vectors from each cluster; and

using an artificial intelligence model, comprising a plurality of weights, convert each sampled input mean vectors to a patch-level score.

3. The method of claim 2 further comprising:

selecting a first group of patches having highest patch-level scores amongst the patch-level scores;

selecting a second group of patches having lowest patch-level scores amongst the path-level scores;

combining the first and second groups; and

generating the patient-level score.

4. The method of claim 3 further comprising:

comparing the patient-level score for a patient to the patient outcome data; and

if the patient-level score does not align with the patient outcome data modify the weights of the artificial intelligence model.

5. The method of claim 1 further comprising identifying regions of interest in the patient image slide, at least partly based on the patch-level scores, wherein the regions of interest comprise biomarkers predictive of patient outcome.

6. The method of claim 5 further comprising:

performing molecular analysis on the regions of interest; and

identifying biomarkers based at least partly on differential expressions of biological macromolecules, comprising DNA, RNA or proteins, on the regions of interest versus other regions of the patient tissue image slide, wherein the biomarkers are over or under expressed in the regions of interest versus other regions.

7. The method of claim 5 further comprising:

generating a biomarker slide based on the identified biomarkers;

generating a patient image slide annotated with the regions of interest; and

co-registering the biomarker slide and the annotated patient tissue image slide.

8. The method of claim 1 , wherein generating labeled patch vectors further comprises:

receiving a plurality of auxiliary labels; and

when an auxiliary label is applicable to a patch vector or its corresponding patch, determine the label of the patch by processing the patch through the artificial intelligence network at a plurality of distinct sizes and/or resolutions.

9. The method of claim 1 , further comprising:

clustering the input training data of the artificial intelligence network based, at least partly, on the output vectors of the artificial intelligence network, wherein the output vectors of the artificial intelligence network indicate an underlying morphological pattern in the patient image slide; and

sampling uniformly across the clusters of the input training data in subsequent training passes of the artificial intelligence network.

10. The method of claim 1 , further comprising:

assigning labels to patch vectors based, at least partly, on a confidence level;

determining label assignments having a low confidence level;

determining input patch vectors generating the low confidence level assignments; and

sampling input training data of the artificial intelligence network to include a pre-determined percentage of input training data from the input patch vectors having generated low confidence level assignments in previous training passes of the artificial intelligence network.

11. A system of predicting patient response to therapy, comprising:

a patch generator configured to receive a plurality of patient tissue image slides and divide the image slides into patches;

a disease detection and grading module, comprising an artificial intelligence network, wherein the disease detection and grading module is configured to:

receive patient outcome data;

receive a plurality of labels correlated with a disease or lack thereof, wherein each label comprises a morphological type, wherein a morphological type corresponds to molecular changes associated with the disease or lack of molecular changes associated with lack of the disease;

covert each patch to a patch vector;

train the artificial intelligence network, based on the plurality of labels to identify the morphological types associated with each label in the image slides;

generate labeled patch vectors, wherein the label of a patch vector is assigned based, at least partly, on the plurality of the labels and the morphological type expressed in the patch;

a morphology detector comprising an unsupervised machine learning model configured to cluster the labeled patch vectors, within each label, wherein clustering groups the path vectors within each label to one or more morphological subtypes expressed in the patch corresponding to the labeled patch vector and a patient outcome, wherein clustering comprises clustering similar patch vectors within each label, and wherein morphological subtypes comprise unknown morphological sub-patterns within a morphological type; and

a region of interest and outcome prediction module configured to:

receive the clustered labeled vectors and generate a patch-level score for each patch based at least partly on the cluster to which the patch vector of the patch belongs;

generate a patient-level score for each patient, at least partly based on the patch-level scores generated for each patient; and

predict a patient response to a therapy, based at least in part, on the patient-level score for the patient.

12. The system of claim 11 , wherein generating the patch-level score further comprises:

converting regions surrounding each patch to microenvironment vectors;

obtaining mean vectors by averaging the patch vectors corresponding to the patches with the microenvironment vectors;

clustering the mean vectors;

sampling input mean vectors from each cluster; and

using an artificial intelligence model, comprising a plurality of weights, convert each sampled input mean vectors to a patch-level score.

13. The system of claim 12 , wherein the region of interest and outcome prediction module is further configured to perform steps comprising:

selecting a first group of patches having highest patch-level scores amongst the patch-level scores;

selecting a second group of patches having lowest patch-level scores amongst the path-level scores;

combining the first and second groups; and

generating the patient-level score.

14. The system of claim 13 , wherein the region of interest and outcome prediction module is further configured to perform steps comprising:

comparing the patient-level score for a patient to the patient outcome data; and

if the patient-level score does not align with the patient outcome data modify the weights of the artificial intelligence model.

15. The system of claim 11 , wherein the region of interest and outcome prediction module is further configured to identify regions of interest in the patient tissue image slide, at least partly based on the patch-level scores, wherein the regions of interest comprise biomarkers predictive of patient outcome.

16. The system of claim 15 further comprising of a spatial profiling and biomarker identification module configured to:

perform molecular analysis on the regions of interest; and

identify biomarkers based at least partly on differential expressions of one or of DNA, RNA or proteins on the regions of interest versus other regions of the patient tissue image slide, wherein the biomarkers are over- or under-expressed in the regions of interest versus other regions, wherein molecular analysis reveals biologically discrete subsets and pathways and mechanism-related response and outcome.

17. The system of claim 15 further comprising a co-registration module configured to:

generate a biomarker slide based on the identified biomarkers;

generate a patient tissue image slide annotated with the regions of interest; and

superimpose the biomarker slide and the annotated patient tissue image slide.

18. The system of claim 11 , wherein the disease detection and grading module is further configured to generate labeled patch vectors by:

receiving a plurality of auxiliary labels; and

when an auxiliary label is applicable to a patch vector or its corresponding patch, determine the label of the patch by processing the patch through the artificial intelligence network at a plurality of distinct sizes and/or resolutions.

19. The system of claim 11 , wherein the disease detection and grading module is further configured to:

cluster the input training data of the artificial intelligence network based, at least partly, on the output vectors of the artificial intelligence network, wherein the output vectors of the artificial intelligence network indicate an underlying morphological pattern in the patient image slide; and

sample uniformly across the clusters of the input training data in subsequent training passes of the artificial intelligence network.

20. The system of claim 11 , wherein the disease detection and grading module is further configured to:

assign labels to patch vectors based, at least partly, on a confidence level;

determine label assignments having a low confidence level;

determine input patch vectors generating the low confidence level assignments; and

sample input training data of the artificial intelligence network to include a pre-determined percentage of input training data from the input patch vectors having generated low confidence level assignments in previous training passes of the artificial intelligence network.