Systems and methods for processing electronic images to infer biomarkers
Systems and methods are disclosed for receiving a target electronic image corresponding to a target specimen, the target specimen comprising a tissue sample of a patient, applying a machine learning system to the target electronic image to identify a region of interest of the target specimen and determine an expression level of, category of, and/or presence of a biomarker in the region of interest, the biomarker comprising at least one from among an epithelial growth factor receptor (EGFR) biomarker and/or a DNA mismatch repair (MMR) deficiency biomarker, the machine learning system having been generated by processing a plurality of training images to predict whether a region of interest is present in the target electronic image, the training images comprising images of human tissue and/or images that are algorithmically generated, and outputting the determined expression level of, category of, and/or presence of the biomarker in the region of interest.
1. A computer-implemented method for analyzing a plurality of digital images corresponding to a pathology specimen, comprising:
receiving the plurality of digital images of the pathology specimen;
determining, by a machine learning system, a human epidermal growth factor receptor 2 (HER2) biomarker expression level prediction for the plurality of digital images, the machine learning system having been trained by processing a plurality of training images wherein determining the HER2 biomarker expression level prediction includes:
breaking each of the plurality of digital images into a plurality of tiles;
determining, by machine learning, a HER2 score corresponding to a prediction for each tile to determine a plurality of tile predictions, each HER2 score being based on immunohistochemistry (IHC) on a scale of 0, 1+, 1+ to 2+, 2+, and/or 3+; and
aggregating the plurality of tile predictions into at least one part level HER2 prediction, the HER2 biomarker expression level prediction being based on the at least one part level HER2 prediction; and
outputting, based on the HER2 biomarker expression level prediction, an HER2 score.
2. The method of claim 1 , wherein determining, by the machine learning system, the HER2 biomarker expression level prediction further comprises:
analyzing histologic and immunohistochemistry (IHC) patient samples to characterize one or more constellations of morphologic features that distinguish HER2 biomarker levels.
3. The method of claim 1 , wherein determining the HER2 biomarker expression level prediction comprises:
predicting at a tile level, by the machine learning system, a probability of a presence of one or more HER2 biomarkers at a level of HER2-low, HER2-high, 0, 1, 2, or 3;
computing, based on the predicted probability of HER biomarkers, a part level score;
outputting, based on the part level score, a plurality of HER2 score predictions; and
aggregating the HER2 score predictions across the plurality of tiles to classify cancer probability in one or more of the plurality of tiles.
4. The method of claim 1 , wherein determining the HER2 biomarker expression level prediction comprises:
predicting at a tile level, by the machine learning system, a probability of a presence of HER2 biomarkers at a level of HER2-low, HER2-high, 0, 1, 2, or 3;
classifying, based on the probability, HER2 biomarkers as positive at levels of approximately 1 to 2; and
determining a recommended treatment decision to administer one or more anti-HER2 antibodies based on the classifying of the HER2 biomarkers being positive, the one or more anti-HER2 antibodies comprising trastuzumab.
5. The method of claim 4 , further comprising:
upon determining that the probability of the tiles are greater than a predetermined threshold, outputting that one or more of the tiles identify an invasive cancer.
6. The method of claim 5 , wherein the predetermined threshold is based on an area under a receiver operating characteristic curve value for a prediction of HER2 scores.
7. The method of claim 1 , wherein determining the HER2 biomarker expression level prediction comprises using classifier training with a convolutional neural network (CNN) to predict a probability that cancer is present in tiles created from the plurality of digital images.
8. The method of claim 1 , wherein the plurality of digital images are stained using Hematoxylin and Eosin (H&E).
9. The method of claim 1 , wherein the pathology specimen is a breast cancer specimen.
10. A system for analyzing a plurality of digital images corresponding to a pathology specimen, comprising:
at least one memory storing instructions; and
at least one processor configured to execute instructions to perform operations comprising:
receiving the plurality of digital images of the pathology specimen;
determining, by a machine learning system, a human epidermal growth factor receptor 2 (HER2) biomarker expression level prediction for an expression level of the plurality of digital images, the machine learning system having been trained by processing a plurality of training images;
predicting, at a tile level of the plurality of digital images, by the machine learning system, a probability of a presence of HER2 biomarkers at a level of HER2-low, HER2-high, 0, 1, 2, or 3, wherein predicting the probability comprises:
computing, based on the probability, a part level score;
outputting, based on the part level score, a plurality of HER2 score predictions; and
aggregating the HER2 score predictions across the plurality of tiles to classify cancer probability at the tile level;
classifying, based on the probability, HER2 biomarkers as positive at one or more levels; and
generating, based on classifying the HER2 biomarkers as positive, a treatment recommendation comprising at least administration of one or more anti-HER2 antibodies.
11. The system of claim 10 , wherein determining, by the machine learning system, the HER2 biomarker expression level prediction comprises analyzing histologic and immunohistochemistry (IHC) patient samples data to characterize one or more constellations of morphologic features that distinguish HER2 biomarker levels.
12. The system of claim 10 , wherein the plurality of digital images are stained using Hematoxylin and Eosin (H&E).
13. The system of claim 10 , wherein the pathology specimen is a breast cancer specimen.
14. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations for analyzing a plurality of digital images corresponding to a pathology specimen, the operations comprising:
receiving the plurality of digital images of the pathology specimen;
determining, by a machine learning system, a human epidermal growth factor receptor 2 (HER2) biomarker expression level prediction for an expression level of the plurality of digital images by analyzing histologic and immunohistochemistry (IHC) patient samples to characterize one or more constellations of morphologic features that distinguish HER2 biomarker levels wherein determining the HER2 biomarker expression level prediction includes:
breaking each of the plurality of digital images into a plurality of tiles;
determining, by machine learning, a HER2 score corresponding to a prediction for each tile to determine a plurality of tile predictions, each HER2 score being based on immunohistochemistry (IHC) on a scale of 0, 1+, 1+ to 2+, 2+, and/or 3+; and
aggregating the plurality of tile predictions into at least one part level HER2 prediction, the HER2 biomarker expression level prediction being based on the at least one part level HER2 prediction; and
outputting, based on the HER2 biomarker expression level prediction, the determined expression level of an HER2 score.
15. The computer-readable medium of claim 14 , wherein the plurality of digital images are stained using Hematoxylin and Eosin (H&E).
16. The computer-readable medium of claim 14 , wherein determining the HER2 biomarker expression level prediction comprises:
predicting at a tile level, by the machine learning system, a probability of a presence of HER2 biomarkers at a level of HER2-low, HER2-high, 0, 1, 2, or 3;
classifying, based on the probability, HER2 biomarkers as positive at levels of approximately 1 to 2; and
determining a recommended treatment decision to administer one or more anti-HER2 antibodies based on the classifying the HER2 biomarkers being positive, the one or more anti-HER2 antibodies comprising trastuzumab.
17. The computer-readable medium of claim 16 , the operations further comprising: upon determining that the probability of the tiles are greater than a predetermined threshold, outputting that one or more of the tiles identify an invasive cancer, wherein the predetermined threshold is based on an area under a receiver operating characteristic curve value for a prediction of HER2 scores.