MEDICAL SPECTROSCOPY AND IMAGING ANALYSIS
Systems and methods for medical imaging and spectroscopy analysis are disclosed. Some embodiments relate to digital staining. Some embodiments relate to digital staining using hyperspectral or multispectral imaging. Some embodiments relate to digital staining using RGB imaging. Some embodiments relate to analysis of other types of medical imaging and spectroscopy. Some embodiments relate to platform for performing analysis of medical imaging and spectroscopy data.
1 . A computer system for generating a model for electronically generating a digitally stained medical image of a tissue sample, the computer system comprising:
a camera;
one or more processors; and
an electronic storage medium,
the camera configured to receive first visible light through an aperture of the camera;
the camera configured to generate a first image comprising red, green, and blue channels from the received first visible light, wherein the first image is a first RGB image of an unstained tissue sample;
the camera configured to receive second visible light through the aperture of the camera;
the camera configured to generate a second image comprising red, green, and blue channels from the received second visible light, wherein the second image is a second RGB image of a stained tissue sample,
wherein the unstained tissue sample and the stained tissue comprise a same tissue;
the camera is in electronic communication with the one or more processors and the electronic storage medium;
the camera configured to electronically store the first and second images in the electronic storage medium;
the electronic storage medium comprises instructions that, when executed by the one or more processors, cause the one or more processors to:
execute first registration instructions comprising:
determining a first difference between the first image and the second image;
modifying one or more of the first image and the second image, wherein the modifying comprises one or more of rotation, translation, or deformation;
determining a second difference between first image and the second image;
determining that the second difference between the first image and second image is within an acceptable threshold value; and
generating a co-registered image pair comprising an unstained image and a ground truth image, wherein the unstained image comprises the first image or the modified first image and the ground truth image comprises the second image or the modified second image;
execute a first model training process comprising:
digitally staining the unstained image to generate a first digitally stained image; and
computing a loss function, wherein the loss function considers a subset of ground truth image data and a subset of digitally stained image data considers the differences between individual pixels of the ground truth image and the first digitally stained image and differences in a spatial distribution of colors in the ground truth image and the first digitally stained image; and
based at least in part on a result of the loss function, adjusting one or more weights of the model;
determining that an output of the loss function is within a threshold amount;
execute second registration instructions comprising:
determining a third difference between the first digitally stained image generated by the trained model and the ground truth image;
modifying one or more of the unstained image and the ground truth image, wherein the modifying comprises one or more of rotation, translation, or deformation;
determining a fourth difference between the first digitally stained image and the ground truth image;
determining that the fourth difference is less within another acceptable threshold value; and
generating a second co-registered image pair comprising a second ground truth image comprising the ground truth image or the modified ground truth image and a second unstained image comprising the unstained image or the modified unstained image;
execute a second model training process comprising:
digitally staining the second unstained image to generate a second digitally stained image;
computing a second loss function, wherein the second loss function considers a subset of second ground truth image data and a subset of second digitally stained image data; and
based on a result of the second loss function, adjusting one or more weights of the model; and
store a generated digital staining model generated by the first model training process and the second model training process in the electronic storage medium, wherein the generated digital staining model comprises the one or more weights.