Devices, systems, and methods for virtual staining
The disclosure herein provides methods, systems, and devices for virtually staining biological tissue for enhanced visualization without use of an actual dye or tag by detecting how each pixel of an unstained tissue image changes in waveform after staining with a certain dye(s) and/or tag(s) or other transformation under a certain electromagnetic radiation source, developing a virtual staining transform based on such detection, and applying such virtual staining transform to an unstained biological tissue to virtually stain the tissue.
1. An apparatus for providing medical therapy to an in vivo tissue sample, the apparatus comprising:
a medical probe having a virtual staining device configured to provide in vivo virtual staining of the in vivo tissue sample, the virtual staining device comprising an electromagnetic radiation source for directing an electromagnetic radiation at the in vivo tissue sample, and a hyperspectral detector for detecting electromagnetic radiation reflected or transmitted from the in vivo tissue sample,
a communication interface configured to provide communication between the electromagnetic radiation source and a computer system, or the hyperspectral detector and the computer system;
an initial image generation module of the computer system configured to generate an input image from the detected electromagnetic radiation from the in vivo tissue sample, wherein the input image comprises a plurality of pixels;
a virtual staining module of the computer system configured to assign a virtual stain to the input image of the in vivo tissue sample based on an identified waveform associated with each one of a plurality of pixels forming the input image;
a therapeutic element configured to provide the medical therapy to the in vivo tissue sample based at least in part on the virtual stain; and
an output image generation module of the computer system configured to generate a virtually stained image based on the assigned virtual stain to the input image wherein the virtually stained image is substantially identical to an image of the in vivo tissue sample when the in vivo tissue sample is treated with an actual stain that corresponds to the virtual stain.
2. The apparatus of claim 1 , wherein the therapeutic element is incorporated into the medical probe.
3. The apparatus of claim 1 , wherein the actual stain is at least one of a dye configured to color certain portions of the in vivo tissue sample and a tag or probe.
4. The apparatus of claim 3 , wherein the tag or probe is at least one of the group comprising an antibody, an aptamer, and a fluorescent protein.
5. A computer-implemented method for virtually staining a tissue sample, the computer-implemented method comprising:
obtaining by a computing system an electronic image of the tissue sample, the electronic image generated by a hyperspectral detector;
determining by the computing system a vector signature or waveform signature associated with each pixel in the electronic image;
generating by the computing system an output pixel for each pixel in the electronic image based on inputting the determined vector signature or waveform signature into a virtual staining transform; and
outputting by the computing system a virtually stained image of the tissue sample based on the generated output pixels, the virtually stained image is substantially identical to an image of the tissue sample when the tissue sample is treated with an actual stain that corresponds to a virtual stain defined by the virtual staining transform,
wherein the computing system comprises a computer processor and an electronic storage medium.
6. The computer-implemented method of claim 5 , wherein the computing system comprises one or more computer systems.
7. The computer-implemented method of claim 5 , wherein the actual stain is a dye configured to color certain portions of the tissue sample.
8. The computer-implemented method of claim 5 , wherein the actual stain is a tag or probe.
9. The computer-implemented method of claim 8 , wherein the tag or probe is at least one of the group comprising an antibody, an aptamer, and a fluorescent protein.
10. The computer-implemented method of claim 5 , wherein the method is performed in vivo.
11. The computer-implemented method of claim 5 , wherein the method is performed in vitro.
12. A system for virtually staining a tissue sample, the system comprising a storage computer system comprising a computer processor configured to execute modules comprising at least:
a data receiving module configured to obtain electronically an electronic image of the tissue sample, the electronic image of the tissue sample generated by a hyperspectral detector;
a pixel analysis module configured to determine a vector signature or waveform signature associated with each pixel in the electronic image;
a virtual transform module configured to generate an output pixel for each pixel in the electronic image based on inputting the determined vector signature or waveform signature into a virtual staining transform; and
an output module configured to output a virtually stained image of the tissue sample based on the generated output pixels,
the virtually stained image is substantially identical to an image of the tissue sample when the tissue sample is treated with an actual stain that corresponds to a virtual stain defined by the virtual staining transform.
13. The system of claim 12 , wherein the actual stain is a dye configured to color certain portions of the tissue sample.
14. The system of claim 12 , wherein the actual stain is a tag or probe.
15. The system of claim 14 , wherein the tag or probe is at least one of the group comprising an antibody, an aptamer, and a fluorescent protein.
16. The system of claim 12 , wherein the electronic image is obtained in vivo.
17. The system of claim 12 , wherein the electronic image is obtained in vitro.