Systems and methods for training a statistical model to predict tissue characteristics for a pathology image
In some aspects, the described systems and methods provide for a method for training a statistical model to predict tissue characteristics for a pathology image. The method includes accessing annotated pathology images. Each of the images includes an annotation describing a tissue characteristic category for a portion of the image. A set of training patches and a corresponding set of annotations are defined using an annotated pathology image. Each of the training patches in the set includes values obtained from a respective subset of pixels in the annotated pathology image and is associated with a corresponding patch annotation determined based on an annotation associated with the respective subset of pixels. The statistical model is trained based on the set of training patches and the corresponding set of patch annotations. The trained statistical model is stored on at least one storage device.
1. A method for predicting an entity of interest for a pathology image, the method comprising:
accessing a pathology image;
retrieving a first trained statistical model from at least one storage device, wherein the first trained statistical model is trained on a plurality of annotated pathology images, wherein each image of the plurality of annotated pathology images includes at least one annotation describing tissue characteristics for one or more portions of the image;
processing, using the first trained statistical model, the pathology image to generate an annotated pathology image including one or more annotations for the pathology image;
extracting values for one or more features from the annotated pathology image;
retrieving a second trained statistical model from the at least one storage device, wherein the second trained statistical model is trained on extracted values for the one or more features from the plurality of annotated pathology images;
processing, using the second trained statistical model, the values for the one or more features extracted from the annotated pathology image to predict an entity of interest, wherein the entity of interest is selected from a group consisting of survival time, drug response, patient level phenotype/molecular characteristics, mutational burden, tumor molecular characteristics, transcriptomic features, protein expression features, and patient clinical outcomes; and
storing the predicted entity of interest on the at least one storage device.
2. The method of claim 1 , wherein the first trained statistical model comprises a convolutional neural network including a plurality of layers, wherein there is no padding applied to an output of any layer of the plurality of layers.
3. The method of claim 1 , wherein the first trained statistical model comprises a convolutional neural network including a plurality of layers, wherein at least one layer of the plurality of layers is aligned such that (N−K)/S is an integer, wherein N represents a size of each input dimension of the at least one layer, wherein K represents a size of a convolution filter of the at least one layer, and wherein S represents a size of a stride of the at least one layer.
4. The method of claim 1 , wherein processing, using the first trained statistical model, the pathology image to generate the annotated pathology image including the one or more annotations for the pathology image comprises:
associating one or more portions of the pathology image with the predicted one or more annotations; and
storing the associations of the one or more portions of the pathology image with the predicted one or more annotations on the at least one storage device.
5. The method of claim 1 , wherein the one or more features are selected from a group consisting of area of epithelium, area of stroma, area of necrosis, area of cancer cells, area of macrophages, area of lymphocytes, number of mitotic figures, average nuclear grade, average distance between fibroblasts and lymphocytes, average distance between immunohistochemistry-positive macrophages and cancer cells, standard deviation of nuclear grade, average distance between blood vessels and tumor cells.
6. The method of claim 1 , wherein the second trained statistical model comprises a generalized linear model, a random forest, a support vector machine, and/or a gradient boosted tree.
7. The method of claim 1 , wherein extracting the values for the one or more features from the annotated pathology image comprises applying one or more functions to the annotated pathology image to extract the values for the one or more features.
8. The method of claim 1 , further comprising determining a first value for a first feature based on a combination of at least some of the values for the one or more features.
9. The method of claim 1 , wherein the annotated pathology image includes a heat map.
10. The method of claim 9 , wherein extracting the values for the one or more features from the annotated pathology image comprises applying one or more functions to the heat map to extract the values for the one or more features.
11. A system for predicting an entity of interest for a pathology image, the system comprising:
at least one computer hardware processor; and
at least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by the at least one computer hardware processor, cause the at least one computer hardware processor to perform:
accessing a pathology image;
retrieving a first trained statistical model from at least one storage device, wherein the first trained statistical model is trained on a plurality of annotated pathology images, wherein each image of the plurality of annotated pathology images includes at least one annotation describing tissue characteristics for one or more portions of the image;
processing, using the first trained statistical model, the pathology image to generate an annotated pathology image including one or more annotations for the pathology image;
extracting values for one or more features from the annotated pathology image;
retrieving a second trained statistical model from the at least one storage device, wherein the second trained statistical model is trained on extracted values for the one or more features from the plurality of annotated pathology images;
processing, using the second trained statistical model, the values for the one or more features extracted from the annotated pathology image to predict an entity of interest, wherein the entity of interest is selected from a group consisting of survival time, drug response, patient level phenotype/molecular characteristics, mutational burden, tumor molecular characteristics, transcriptomic features, protein expression features, and patient clinical outcomes; and
storing the predicted entity of interest on the at least one storage device.
12. The system of claim 11 , wherein the first trained statistical model comprises a convolutional neural network including a plurality of layers, wherein there is no padding applied to an output of any layer of the plurality of layers.
13. The system of claim 11 , wherein the first trained statistical model comprises a convolutional neural network including a plurality of layers, wherein at least one layer of the plurality of layers is aligned such that (N−K)/S is an integer, wherein N represents a size of each input dimension of the at least one layer, wherein K represents a size of a convolution filter of the at least one layer, and wherein S represents a size of a stride of the at least one layer.
14. The system of claim 11 , wherein processing, using the first trained statistical model, the pathology image to generate the annotated pathology image including the one or more annotations for the pathology image comprises:
associating one or more portions of the pathology image with the predicted one or more annotations; and
storing the associations of the one or more portions of the pathology image with the predicted one or more annotations on the at least one storage device.
15. The system of claim 11 , wherein the one or more features are selected from a group consisting of area of epithelium, area of stroma, area of necrosis, area of cancer cells, area of macrophages, area of lymphocytes, number of mitotic figures, average nuclear grade, average distance between fibroblasts and lymphocytes, average distance between immunohistochemistry-positive macrophages and cancer cells, standard deviation of nuclear grade, average distance between blood vessels and tumor cells.
16. The system of claim 11 , wherein the second trained statistical model comprises a generalized linear model, a random forest, a support vector machine, and/or a gradient boosted tree.
17. The system of claim 11 , wherein extracting the values for the one or more features from the annotated pathology image comprises applying one or more functions to the annotated pathology image to extract the values for the one or more features.
18. The system of claim 11 , wherein the processor-executable instructions further cause the at least one computer hardware processor to perform determining a first value for a first feature based on a combination of at least some of the values for the one or more features.
19. The system of claim 11 , wherein the annotated pathology image includes a heat map.
20. The system of claim 19 , wherein extracting the values for the one or more features from the annotated pathology image comprises applying one or more functions to the heat map to extract the values for the one or more features.