Determining biomarkers from histopathology slide images
A system for identifying biomarkers in a digital image of a Hematoxylin and Eosin-stained slide of a target tissue includes a processor and an electronic network; and a memory having stored thereon computer-executable instructions that, when executed by the one or more processors, cause the computing system to: process segmented tile images determine a predicted biomarker presence; and transmit the predicted presence. A non-transitory computer-readable medium includes a set of computer-executable instructions that, when executed by one or more processors, cause a computer to: process segmented tile images; determine a predicted biomarker presence; and transmit the predicted presence. A computer-implemented method includes processing segmented tile images; determining a predicted biomarker presence; and transmitting the predicted presence.
1. A computing system for identifying biomarkers in a digital image of a Hematoxylin and Eosin-stained slide of a target tissue, comprising:
one or more processors;
an electronic network; and
one or more memories having stored thereon computer-executable instructions that, when executed by the one or more processors, cause the computing system to:
process a plurality of segmented tile images each corresponding to a different respective portion of the digital image using a deep learning framework by:
(i) predicting a respective biomarker classification for each tile image using one or more trained biomarker classification models; and
(ii) predicting a respective tissue classification for each tile image using one or more trained deep learning classifier models;
determine, based on (i) and (ii), a predicted presence of one or more biomarkers in the target tissue;
transmit, via the electronic network, the predicted presence of the one or more biomarkers;
receive a molecular training dataset for a plurality of training tissue samples, the molecular training dataset comprising molecular data based on sequencing of a substantially similar sample associated with each training tissue sample; and
identify one or more molecular data subsets in the molecular training dataset, each corresponding to a different respective biomarker, by processing the molecular training dataset using a clustering algorithm.
2. The computing system of claim 1 , the one or more memories having stored thereon further instructions that, when executed by the one or more processors, cause the computing system to:
separate the digital image into the plurality of segmented tile images by processing the digital image using at least one of (i) a tiling mask or (ii) a trained multiple instance learning controller.
3. The computing system of claim 1 , the one or more memories having stored thereon further instructions that, when executed by the one or more processors, cause the computing system to:
receive, at the deep learning framework, at least one training Hematoxylin and Eosin-stained slide image having a respective label corresponding to a respective biomarker;
classify the Hematoxylin and Eosin-stained slide image using tile-based tissue classification analysis; and
analyzing the Hematoxylin and Eosin-stained slide image using a pixel-based cell segmentation.
4. The computing system of claim 3 , the one or more memories having stored thereon further instructions that, when executed by the one or more processors, cause the computing system to:
identify a plurality of cells within the plurality of tile images using a trained cell segmentation model by:
applying each of the plurality of tile images to a cell segmentation model and, for each tile image, assigning a cell classification to one or more pixels within the tile image.
5. The computing system of claim 4 , the one or more memories having stored thereon further instructions that, when executed by the one or more processors, cause the computing system to:
assign the cell classification to one or more pixels within the tile image by:
identifying the one or more pixels as a cell interior, a cell border, or a cell exterior; and
classifying the one or more pixels as the cell interior, the cell border, or the cell exterior.
6. The computing system of claim 4 , wherein the trained cell segmentation model is a pixel-resolution three-dimensional classification model trained to classify a cell interior, a cell border, and a cell exterior.
7. The computing system of claim 3 , the one or more memories having stored thereon further instructions that, when executed by the one or more processors, cause the computing system to:
classify the Hematoxylin and Eosin-stained image using tile-based biomarker classification analysis.
8. The computing system of claim 3 , the one or more memories having stored thereon further instructions that, when executed by the one or more processors, cause the computing system to:
generate one or both of (i) the trained biomarker classification models, and (ii) the trained deep learning classifier models.
9. The computing system of claim 1 , the one or more memories having stored thereon further instructions that, when executed by the one or more processors, cause the computing system to:
for each tile image in the plurality of tile images:
infer a class status of the tile image; and
discard, when the class status of the tile image does not correspond to a desired class, the tile image.
10. The computing system of claim 1 ,
wherein at least one of the trained deep learning classifier models is a tile-resolution Fully Convolutional Network (FCN) classification model.
11. The computing system of claim 1 , wherein the one or more biomarkers include at least one of a tumor-infiltrating lymphocyte (TIL) biomarker, a nucleus-to-cytoplasm (NC) ratio biomarker, a ploidy biomarker, a signet ring morphology biomarker, a programmed death-ligand 1 (PD-L1) biomarker, a consensus molecular subtype (CMS) biomarker, a human epidermal growth factor receptor 2 (HER2) biomarker, or a homologous recombination deficiency (HRD) biomarker.
12. The computing system of claim 1 , wherein the deep learning framework includes at least one of a multi-scale deep learning framework or a single-scale deep learning framework.
13. The computing system of claim 12 , wherein the single-scale deep learning framework is a convolution neural network having a ResNet configuration or an Inception configuration.
14. The computing system of claim 1 , the one or more memories having stored thereon further instructions that, when executed by the one or more processors, cause the computing system to:
for each tile image in the plurality of tile images:
process the tile image using a biomarker classification model trained to predict a different respective biomarker classification; and
determine, based on the predicted biomarkers of the tile image, a predicted presence of one or more biomarkers in the target tissue; and
generate a report containing the digital image and a digital overlay visualizing the predicted presence of the one or more biomarkers.
15. The computing system of claim 14 , wherein the digital overlay includes an overlay element identifying tumor content of the digital image or tumor percentage of the digital image.
16. The computing system of claim 1 , the one or more memories having stored thereon further instructions that, when executed by the one or more processors, cause the computing system to:
for each molecular data subset in the one or more molecular data subsets:
receive a plurality of digital images of Hematoxylin and Eosin-stained training slides of training tissue samples corresponding to the respective different biomarker of the molecular data subset in an image-based biomarker prediction system having one or more processors; and
generate one of the trained biomarker classification models, based on the plurality of digital images of the Hematoxylin and Eosin-stained training slides.
17. The computing system of claim 1 , wherein the computing system further comprises:
a pathology slide scanner system;
and the one or more memories have stored thereon further instructions that, when executed by the one or more processors, cause the computing system to:
receive, via the electronic network, the digital image from the pathology slide scanner system.
18. A non-transitory computer-readable medium comprising a set of computer-executable instructions that, when executed by one or more processors, cause a computer to:
process a plurality of segmented tile images each corresponding to a different respective portion of a digital image of a Hematoxylin and Eosin-stained slide of a target tissue using a deep learning framework by:
(i) predicting a respective biomarker classification for each tile image using one or more trained biomarker classification models; and
(ii) predicting a respective tissue classification for each tile image using one or more trained deep learning classifier models;
determine, based on (i) and (ii), a predicted presence of one or more biomarkers in a target tissue;
transmit the predicted presence of the one or more biomarkers;
receive a molecular training dataset for a plurality of training tissue samples, the molecular training dataset comprising molecular data based on sequencing of a substantially similar sample associated with each training tissue sample; and
identify one or more molecular data subsets in the molecular training dataset, each corresponding to a different respective biomarker, by processing the molecular training dataset using a clustering algorithm.
19. A computer-implemented method for identifying biomarkers in a digital image of a Hematoxylin and Eosin-stained slide of a target tissue, comprising:
processing, via one or more processors, a plurality of segmented tile images each corresponding to a different respective portion of the digital image using a deep learning framework by:
(i) predicting a respective biomarker classification for each tile image using one or more trained biomarker classification models; and
(ii) predicting a respective tissue classification for each tile image using one or more trained deep learning classifier models;
determining, based on (i) and (ii), a predicted presence of one or more biomarkers in a target tissue;
transmitting, via an electronic network, the predicted presence of the one or more biomarkers;
receiving a molecular training dataset for a plurality of training tissue samples, the molecular training dataset comprising molecular data based on sequencing of a substantially similar sample associated with each training tissue sample; and
identifying one or more molecular data subsets in the molecular training dataset, each corresponding to a different respective biomarker, by processing the molecular training dataset using a clustering algorithm.