Specimen imaging with x-ray and optical measurement
A surgical specimen imaging system includes a micro-X-ray computed tomography (CT) unit for CT imaging of the specimen and a structured light imaging (SLI) unit for optical imaging at multiple wavelengths, multiple phase offsets, and multiple structured-light pattern periods including unstructured light. The system's image processing unit receives CT and optical images and is configured by firmware in memory to co-register the images and process the optical images to determine texture at multiple subimages of the optical images, determined textures forming a texture map. The texture map is processed by a machine-learning-based classifier to determine a tissue type map of the specimen, and the tissue type map is processed with the CT images to give a 3D tissue-type map. In embodiments, the firmware extracts optical properties including scattering and absorption at multiple wavelengths and the classifier also uses these properties in generating the tissue type map.
1. A system for specimen imaging with combined x-ray and optical measurement comprising:
a micro-X-ray computed tomography (CT) unit configured to provide voxel-based CT images of the specimen;
a structured light imaging (SLI) unit configured to provide optical images obtained at a plurality of wavelengths, a plurality of structured light phases, and at a plurality of structured-light pattern periods;
an image processing unit configured to receive the CT images and the optical images and configured by firmware in memory to:
demodulate the structured light images;
register the CT and optical images;
extract like-tissue volumes from the CT images to form a volume map;
process the optical images to determine texture at a plurality of subimages of the optical images, the determined textures forming a texture map;
use a first machine-learning-based classifier to determine a surface tissue type map of the specimen from the texture map; and
use a second machine-learning-based classifier to determine a three-dimensional tissue-type map from the surface tissue type map and the extracted like-tissue volumes from the CT images.
2. The system of claim 1 wherein the first and second machine-learning-based classifiers are each selected from the group consisting of support vector machine classifiers, neural network classifiers, and k-nearest-neighbor classifiers.
3. The system of claim 2 where the surface tissue type map distinguishes malignant from benign tissue in the specimen.
4. The system of claim 3 where the image processing unit is further configured to process the optical images to determine optical properties at a plurality of pixels, thereby forming optical-properties images, and where the machine-learning-based classifier uses the optical-properties images in forming the tissue type map.
5. The system of claim 4 where at least one of the structured light periods is 1.37 mm −1 .
6. The system of claim 5 where the structured light phases comprise at least three phase offsets.
7. The system of claim 1 where the firmware further comprises firmware to perform color analysis on the optical images to form a color map, and where the first machine-learning-based classifier is configured to use the color map in producing the surface tissue type map.
8. The system of claim 7 where the tissue-type map includes a map of malignant versus benign tissue.
9. A method of analyzing surgical specimens comprising:
illuminating a surgical specimen with structured light from a structured-light illuminator;
imaging the surgical specimen to provide optical images at a plurality of illumination wavelengths, a plurality of structured light phases, and a plurality of structured-light pattern periods;
demodulating the optical images;
processing the optical images to determine texture at a plurality of subimages of the optical images, the determined textures forming a texture map; and
classifying with a first machine-learning-based classifier to determine a surface tissue type map of the specimen from the texture map;
obtaining X-ray computed tomography (CT) images of the surgical specimen;
registering the CT and optical images;
extracting like-tissue volumes from the CT images to form a volume map; and
classifying with a second machine-learning-based classifier to determine a three-dimensional tissue-type map from the surface tissue type map and the extracted like-tissue volumes from the CT images.
10. The method of claim 9 further comprising performing color analysis on the optical images to form a color map, and where the first machine-learning-based classifier is configured to use the color map in producing the surface tissue type map.
11. The method of claim 10 wherein the first and second machine-learning-based classifiers are each selected from the group consisting of support vector machine classifiers, neural network classifiers, and k-nearest-neighbor classifiers.
12. The method of claim 10 further comprising processing the optical images to determine optical properties at a plurality of pixels, thereby forming optical-properties images, and where the machine-learning-based classifier uses the optical-properties images in forming the tissue type map.
13. A method of performing surgery comprising the method of claim 9 and further comprising performing surgery to remove a cancerous lesion from a patient to provide the surgical specimen, and using the tissue type map to determine whether and how much additional tissue should be removed from the patient prior to ending the surgery.