Virtual staining for tissue slide images
A machine learning predictor model is trained to generate a prediction of the appearance of a tissue sample stained with a special stain such as an IHC stain from an input image that is either unstained or stained with H&E. Training data takes the form of thousands of pairs of precisely aligned images, one of which is an image of a tissue specimen stained with H&E or unstained, and the other of which is an image of the tissue specimen stained with the special stain. The model can be trained to predict special stain images for a multitude of different tissue types and special stain types, in use, an input image, e.g., an H&E image of a given tissue specimen at a particular magnification level is provided to the model and the model generates a prediction of the appearance of the tissue specimen as if it were stained with the special stain. The predicted image is provided to a user and displayed, e.g., on a pathology workstation.
1. A method, comprising:
obtaining an input red-green-blue (“RGB”) image of a tissue specimen;
supplying the input RGB image to a machine learning predictor model trained from a multitude of pairs of aligned RGB images of tissue specimens, one of the RGB images of the pair of RGB images comprising an RGB image of a tissue specimen stained with a special stain, the model trained to predict an RGB image of a tissue specimen stained with the special stain;
with the predictor model, generating a predicted RGB image of the tissue specimen stained with the special stain; and
outputting the predicted RGB image as a virtual RGB image.
2. The method of claim 1 , wherein the input RGB image comprises an image of the tissue specimen in an unstained condition.
3. The method of claim 1 , wherein the input RGB image comprises an image of the tissue specimen stained with routine stain.
4. The method of claim 1 , wherein the special stain comprises an IHC stain.
5. The method of claim 1 , wherein the tissue specimen is of one of the following types: breast tissue, prostate tissue, lymph node tissue and lung tissue.
6. The method of claim 1 , wherein the machine learning predictor model comprises a generative adversarial network.
7. The method of claim 1 , wherein the machine learning predictor model comprises a self-supervised learning neural network.
8. The method of claim 1 , wherein the machine learning predictor model comprises a convolutional neural network.
9. The method of claim 1 , wherein the machine learning predictor model comprises a convolutional neural network for dense segmentation.
10. A system comprising:
a non-transitory computer-readable medium; and
one or more processors communicatively coupled to the non-transitory computer-readable medium, the one or more processors configured to execute processor-executable instructions stored in the non-transitory computer-readable medium to:
obtain an input red-green-blue (“RGB”) image of a tissue specimen;
supply the input RGB image to a machine learning predictor model trained from a multitude of pairs of aligned RGB images of tissue specimens, one of the RGB images of the pair of RGB images comprising an RGB image of a tissue specimen stained with a special stain, the model trained to predict an RGB image of a tissue specimen stained with the special stain;
use the predictor model to generate a predicted RGB image of the tissue specimen stained with the special stain; and
output the predicted RGB image as a virtual RGB image.
11. The system of claim 10 , wherein the input RGB image comprises an image of the tissue specimen in an unstained condition.
12. The system of claim 10 , wherein the input RGB image comprises an image of the tissue specimen stained with a routine stain.
13. The system of claim 10 , wherein the special stain comprises an IHC stain.
14. The system of claim 10 , wherein the tissue specimen is of one of the following types: breast tissue, prostate tissue, lymph node tissue and lung tissue.
15. The system of claim 10 , wherein the machine learning predictor model comprises a generative adversarial network.
16. The system of claim 10 , wherein the machine learning predictor model comprises a self-supervised learning neural network.
17. The system of claim 10 , wherein the machine learning predictor model comprises a convolutional neural network.
18. The system of claim 10 , wherein the machine learning predictor model comprises a convolutional neural network for dense segmentation.
19. A non-transitory computer-readable medium comprising processor-executable instructions configured to cause one or more processors to:
obtain an input red-green-blue (“RGB”) image of a tissue specimen;
supply the input RGB image to a machine learning predictor model trained from a multitude of pairs of aligned RGB images of tissue specimens, one of the RGB images of the pair of RGB images comprising an RGB image of a tissue specimen stained with a special stain, the model trained to predict an RGB image of a tissue specimen stained with the special stain;
use the predictor model to generate a predicted RGB image of the tissue specimen stained with the special stain; and
output the predicted RGB image as a virtual RGB image.