Virtual staining for tissue slide images
A machine learning predictor model is trained to generate a prediction of the appearance of a tissue sample stained with a special stain such as an IHC stain from an input image that is either unstained or stained with H&E. Training data takes the form of thousands of pairs of precisely aligned images, one of which is an image of a tissue specimen stained with H&E or unstained, and the other of which is an image of the tissue specimen stained with the special stain. The model can be trained to predict special stain images for a multitude of different tissue types and special stain types, in use, an input image, e.g., an H&E image of a given tissue specimen at a particular magnification level is provided to the model and the model generates a prediction of the appearance of the tissue specimen as if it were stained with the special stain. The predicted image is provided to a user and displayed, e.g., on a pathology workstation.
1. A method of generating a virtual high-resolution image of a tissue specimen stained with a special stain, comprising the steps of:
obtaining an input high-resolution image;
supplying the input high-resolution image to a machine learning predictor model trained from a multitude of pairs of aligned high-resolution images of tissue specimens, one of the high-resolution images of the pair of high-resolution images comprising a high-resolution image of a tissue specimen stained with the special stain, the model trained to predict a high-resolution image of a tissue specimen stained with the special stain;
with the predictor model, generating a predicted high-resolution image of the tissue specimen stained with the special stain; and
outputting the predicted high-resolution image as the virtual high-resolution image.
2. The method of claim 1 , wherein the input high-resolution image comprises an image of the tissue specimen in an unstained condition.
3. The method of claim 1 , wherein the input high-resolution image comprises an image of the tissue specimen stained with H&E.
4. The method of claim 1 , wherein the special stain comprises an IHC stain.
5. The method of claim 1 , wherein the tissue specimen is of one of the following types: breast tissue, prostate tissue, lymph node tissue and lung tissue.
6. The method of claim 1 , wherein the machine learning predictor model comprises a generative adversarial network.
7. The method of claim 1 , wherein the machine learning predictor model comprises a self-supervised learning neural network.
8. The method of claim 1 , wherein the machine learning predictor model comprises a convolutional neural network.
9. The method of claim 1 , wherein the machine learning predictor model comprises a convolutional neural network for dense segmentation.
10. The method of claim 1 , wherein the aligned pairs of images have edge portions having nulled pixel values.
11. A system comprising:
a non-transitory computer-readable medium; and
one or more processors communicatively coupled to the non-transitory computer- readable medium, the one or more processors configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to:
obtain an input high-resolution image;
supply the input high-resolution image to a machine learning predictor model trained from a multitude of pairs of aligned high-resolution images of tissue specimens, one of the high-resolution images of the pair of high-resolution images comprising a high- resolution image of a tissue specimen stained with a special stain, the model trained to predict a high-resolution image of a tissue specimen stained with the special stain;
use the predictor model to generate a predicted high-resolution image of the tissue specimen stained with the special stain; and
output the predicted high-resolution image as a virtual high-resolution image.
12. The system of claim 11 , wherein the input high-resolution image comprises an image of the tissue specimen in an unstained condition.
13. The system of claim 11 , wherein the input high-resolution image comprises an image of the tissue specimen stained with H&E.
14. The system of claim 11 , wherein the special stain comprises an IHC stain.
15. The system of claim 11 , wherein the machine learning predictor model comprises a generative adversarial network.
16. The system of claim 11 , wherein the machine learning predictor model comprises a self-supervised learning neural network.
17. The system of claim 11 , wherein the machine learning predictor model comprises a convolutional neural network.
18. The system of claim 11 , wherein the machine learning predictor model comprises a convolutional neural network for dense segmentation.