DETERMINING BIOMARKERS FROM HISTOPATHOLOGY SLIDE IMAGES

1 . A computing system for identifying biomarkers in a digital image of a Hematoxylin and Eosin-stained slide of a target tissue, comprising:

one or more processors;

an electronic network; and

one or more memories having stored thereon computer-executable instructions that, when executed by the one or more processors, cause the computing system to:

process a plurality of segmented tile images each corresponding to a different respective portion of the digital image using a deep learning framework by:

(i) predicting a respective biomarker classification for each tile image using one or more biomarker classification models,

wherein the one or more biomarker classification models are trained using a molecular training dataset that (a) corresponds to a plurality of training tissue samples, (b) includes molecular data based on sequencing of a substantially similar sample associated with each training tissue sample, and (c) includes a plurality of molecular data subsets clustered by biomarker, and

(ii) predicting a respective tissue classification for each tile image using one or more trained deep learning classifier models;

determine, based on (i) and (ii), a predicted presence of one or more biomarkers in the target tissue; and

transmit, via the electronic network, the predicted presence of the one or more biomarkers.

2 . The computing system of claim 1 , the one or more memories having stored thereon further instructions that, when executed by the one or more processors, cause the computing system to:

separate the digital image into the plurality of segmented tile images by processing the digital image using at least one of (i) a tiling mask or (ii) a trained multiple instance learning controller.

3 . The computing system of claim 1 , the one or more memories having stored thereon further instructions that, when executed by the one or more processors, cause the computing system to:

receive, at the deep learning framework, at least one training Hematoxylin and Eosin-stained slide image having a respective label corresponding to a respective biomarker;

classify the Hematoxylin and Eosin-stained slide image using tile-based tissue classification analysis; and

analyzing the Hematoxylin and Eosin-stained slide image using a pixel-based cell segmentation.

4 . The computing system of claim 3 , the one or more memories having stored thereon further instructions that, when executed by the one or more processors, cause the computing system to:

identify a plurality of cells within the plurality of tile images using a trained cell segmentation model by:

applying each of the plurality of tile images to a cell segmentation model and, for each tile image, assigning a cell classification to one or more pixels within the tile image.

5 . The computing system of claim 4 , the one or more memories having stored thereon further instructions that, when executed by the one or more processors, cause the computing system to:

assign the cell classification to one or more pixels within the tile image by:

identifying the one or more pixels as a cell interior, a cell border, or a cell exterior; and

classifying the one or more pixels as the cell interior, the cell border, or the cell exterior.

6 . The computing system of claim 4 , wherein the trained cell segmentation model is a pixel-resolution three-dimensional classification model trained to classify a cell interior, a cell border, and a cell exterior.

7 . The computing system of claim 3 , the one or more memories having stored thereon further instructions that, when executed by the one or more processors, cause the computing system to:

classify the Hematoxylin and Eosin-stained image using tile-based biomarker classification analysis.

8 . The computing system of claim 3 , the one or more memories having stored thereon further instructions that, when executed by the one or more processors, cause the computing system to:

generate one or both of (i) the trained biomarker classification models, and (ii) the trained deep learning classifier models.

9 . The computing system of claim 1 , the one or more memories having stored thereon further instructions that, when executed by the one or more processors, cause the computing system to:

for each tile image in the plurality of tile images:

infer a class status of the tile image; and

discard, when the class status of the tile image does not correspond to a desired class, the tile image.

10 . The computing system of claim 1 ,

wherein at least one of the trained deep learning classifier models is a tile-resolution Fully Convolutional Network (FCN) classification model.

11 . The computing system of claim 1 , wherein the one or more biomarkers include at least one of a tumor-infiltrating lymphocyte (TIL) biomarker, a nucleus-to-cytoplasm (NC) ratio biomarker, a ploidy biomarker, a signet ring morphology biomarker, a programmed death-ligand 1 (PD-L1) biomarker, a consensus molecular subtype (CMS) biomarker, a human epidermal growth factor receptor 2 (HER2) biomarker, or a homologous recombination deficiency (HRD) biomarker.

12 . The computing system of claim 1 , wherein the deep learning framework includes at least one of a multi-scale deep learning framework or a single-scale deep learning framework.

13 . The computing system of claim 12 , wherein the single-scale deep learning framework is a convolution neural network having a ResNet configuration or an Inception configuration.

14 . The computing system of claim 1 , the one or more memories having stored thereon further instructions that, when executed by the one or more processors, cause the computing system to:

for each tile image in the plurality of tile images:

process the tile image using a biomarker classification model trained to predict a different respective biomarker classification; and

determine, based on the predicted biomarkers of the tile image, a predicted presence of one or more biomarkers in the target tissue; and

generate a report containing the digital image and a digital overlay visualizing the predicted presence of the one or more biomarkers.

15 . The computing system of claim 14 , wherein the digital overlay includes an overlay element identifying tumor content of the digital image or tumor percentage of the digital image.

16 . The computing system of claim 1 , the one or more memories having stored thereon further instructions that, when executed by the one or more processors, cause the computing system to:

for each molecular data subset in the one or more molecular data subsets:

receive a plurality of digital images of Hematoxylin and Eosin-stained training slides of training tissue samples corresponding to the respective different biomarker of the molecular data subset in an image-based biomarker prediction system having one or more processors; and

generate one of the trained biomarker classification models, based on the plurality of digital images of the Hematoxylin and Eosin-stained training slides.

17 . The computing system of claim 1 , wherein the computing system further comprises:

a pathology slide scanner system;

and the one or more memories have stored thereon further instructions that, when executed by the one or more processors, cause the computing system to:

receive, via the electronic network, the digital image from the pathology slide scanner system.

18 . A non-transitory computer-readable medium comprising a set of computer-executable instructions that, when executed by one or more processors, cause a computer to:

process a plurality of segmented tile images each corresponding to a different respective portion of the digital image using a deep learning framework by:

(i) predicting a respective biomarker classification for each tile image using one or more biomarker classification models,

wherein the one or more biomarker classification models are trained using a molecular training dataset that (a) corresponds to a plurality of training tissue samples, (b) includes molecular data based on sequencing of a substantially similar sample associated with each training tissue sample, and (c) includes a plurality of molecular data subsets clustered by biomarker, and

(ii) predicting a respective tissue classification for each tile image using one or more trained deep learning classifier models;

determine, based on (i) and (ii), a predicted presence of one or more biomarkers in the target tissue; and

transmit, via the electronic network, the predicted presence of the one or more biomarkers.

19 . A computer-implemented method for identifying biomarkers in a digital image of a Hematoxylin and Eosin-stained slide of a target tissue, comprising:

processing a plurality of segmented tile images each corresponding to a different respective portion of the digital image using a deep learning framework by:

(i) predicting a respective biomarker classification for each tile image using one or more biomarker classification models,

wherein the one or more biomarker classification models are trained using a molecular training dataset that (a) corresponds to a plurality of training tissue samples, (b) includes molecular data based on sequencing of a substantially similar sample associated with each training tissue sample, and (c) includes a plurality of molecular data subsets clustered by biomarker, and

(ii) predicting a respective tissue classification for each tile image using one or more trained deep learning classifier models;

determining, based on (i) and (ii), a predicted presence of one or more biomarkers in the target tissue; and

transmitting, via the electronic network, the predicted presence of the one or more biomarkers.