Assessing heterogeneity of features in digital pathology images using machine learning techniques

1 . A computer-implemented method comprising:

receiving a digital pathology image of a tissue sample from a patient;

subdividing the digital pathology image into a plurality of patches;

for each patch of the plurality of patches:

identifying a plurality of image features detected in the patch; and

generating a plurality of labels corresponding to the plurality of image features identified in the patch using a trained machine-learning model, wherein the plurality of labels comprising one or more labels of a first label type and one or more labels of a second label type;

determining, based on a relative frequency of the first label type relative to the second label type in the generated plurality of labels, a heterogeneity metric for the tissue sample, wherein the heterogeneity metric quantifies a degree of heterogeneity of the tissue sample;

generating an assessment of the tissue sample based on the heterogeneity metric, wherein the assessment comprises a prediction of a responsiveness of the patient to one or more treatments;

generating a patch-based signature based on the generated plurality of labels; and

using the patch-based signature, generating a visualization of image features for evaluating the trained machine-learning model.

2 . The method of claim 1 , wherein the digital pathology image of the tissue sample is a whole-slide image of a tumor sample from the patient, wherein the patient is diagnosed with non-small cell lung cancer.

3 . The method of claim 1 , wherein the digital pathology image is a hematoxylin and eosin (H&E) stained image.

4 . The method of claim 1 , wherein the image features detected in the patch of the digital pathology image correspond to histologies.

5 . The method of claim 4 , wherein the generated plurality of labels correspond to adenocarcinoma and squamous cell carcinoma cancer regions.

6 . The method of claim 1 , wherein the image features detected in the plurality of patches of the digital pathology image correspond to mutations or gene variants.

7 . The method of claim 6 , wherein the generated plurality of labels correspond to Kirsten rat sarcoma viral oncogene homolog (KRAS) mutation, epidermal growth factor receptor (EGFR) mutation, anaplastic lymphoma kinase (ALK) mutation, or tumor protein (TP53) mutation.

8 . The method of claim 1 , wherein the heterogeneity metric quantifies a degree of heterogeneity of the identified image features and corresponding labels in the tissue sample.

9 . The method of claim 1 , wherein the patch-based signature includes a heatmap, the heatmap comprising a plurality of regions, each region of the plurality of regions being associated with an intensity value, respectively, wherein one or more regions of the plurality of regions are further associated with a predicted label of the patch of the digital pathology image.

10 . The method of claim 1 , wherein the visualization of the image features identified in the tissue sample comprises displaying each of the generated plurality of labels in differentiated color coding.

11 . The method of claim 1 , wherein the patch-based signature is generated using a saliency mapping technique.

12 . The method of claim 1 , further comprising training the machine-learning model, wherein training the machine-learning model comprises:

accessing a plurality of digital pathology images associated with a plurality of subjects, respectively;

identifying a tumor region in each of the plurality of digital pathology images;

subdividing each of the plurality of digital pathology images into a set of training patches, wherein each training patch in the set is classified with one or more features and annotated with one or more ground-truth labels corresponding to the one or more features; and

using classified set of training patches with ground-truth labels corresponding to the features depicted in the patches to train the machine-learning model.

13 . The method of claim 12 , wherein the ground-truth labels are provided by a clinician.

14 . The method of claim 12 , further comprising updating the machine-learning model, wherein updating the machine-learning model comprises:

accessing a particular digital pathology image of a particular subject;

subdividing the particular digital pathology image into a second set of patches;

identifying second images features in the second set of patches;

generating a set of predicted labels corresponding to the second image features identified in the second set of patches using the trained machine-learning model;

comparing the generated set of predicted labels with the ground-truth labels to measure an accuracy in label prediction of the machine-learning model; and

updating the machine-learning model based on the comparison, wherein updating the machine-learning model comprises further training the machine-learning model.

15 . The method of claim 1 , wherein the machine-learning model is a deep-learning neural network.

16 . The method of claim 1 , further comprising:

determining one or more treatment options for a patient based on the assessment.

17 . A digital pathology image processing system comprising:

one or more processors; and

one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to perform operations comprising:

receiving a digital pathology image of a tissue sample from a patient;

subdividing the digital pathology image into a plurality of patches;

for each patch of the plurality of patches:

identifying a plurality of image features detected in the patch; and

generating a plurality of labels corresponding to the plurality of image features identified in the patch using a trained machine-learning model, wherein the plurality of labels comprising one or more labels of a first label type and one or more labels of a second label type;

determining, based on a relative frequency of the first label type relative to the second label type in the generated plurality of labels, a heterogeneity metric for the tissue sample, wherein the heterogeneity metric quantifies a degree of heterogeneity of the tissue sample;

generating an assessment of the tissue sample based on the heterogeneity metric, wherein the assessment comprises a prediction of a responsiveness of the patient to one or more treatments;

generating a patch-based signature based on the generated plurality of labels; and

using the patch-based signature, generating a visualization of image features for evaluating the machine-learning model.

18 . The digital pathology image processing system of claim 17 , wherein the image features detected in the patch of the digital pathology image correspond to histologies or mutations.

19 . One or more computer-readable non-transitory storage media including instructions that, when executed by one or more processors, are configured to cause the one or more processors of a digital pathology image processing system to perform operations comprising:

receiving a digital pathology image of a tissue sample from a patient;

subdividing the digital pathology image into a plurality of patches;

for each patch of the plurality of patches:

identifying a plurality of image features detected in the patch; and

generating a plurality of labels corresponding to the plurality of image features identified in the patch using a trained machine-learning model, wherein the plurality of labels comprising one or more labels of a first label type and one or more labels of a second label type;

determining, based on a relative frequency of the first label type relative to the second label type in the generated plurality of labels, a heterogeneity metric for the tissue sample, wherein the heterogeneity metric quantifies a degree of heterogeneity of the tissue sample;

generating an assessment of the tissue sample based on the heterogeneity metric, wherein the assessment comprises a prediction of a responsiveness of the patient to one or more treatments;

generating a patch-based signature based on the generated plurality of labels; and

using the patch-based signature, generating a visualization of image features for evaluating the machine-learning model.