Hand-held stereovision system for image updating in surgery
This invention provides a hand-held stereovision (HHS) system that is an efficient, accurate, and radiation-free imaging device to acquire intraoperative profiles of the exposed spine in prone position. The reconstructed intraoperative stereovision surfaces (iSV) are registered with preoperative CT (pCT; supine position) in a nonrigid fashion to generate updated CT images (uCT) and correct for vertebral posture and alignment changes. Updated CT images are uploaded to a commercial navigation system for surgical navigation.
1. A stereoscopic vision system for use in surgery procedures comprising:
a handheld stereovision (HHS) device comprising:
a body;
at least two cameras mounted to the body, the at least two cameras angled towards a center of a field of view, and having an overlapping region of view;
a surgical tracker; and
a processor operatively connected to the HHS device through a data connection, the processor comprising:
a patient registration module configured to register intraoperative stereovision (iSV) surfaces with preoperative CT (pCT) images to generate updated CT (uCT) images for use in a surgical navigation system, wherein registering pCT images with the iSV images comprises non-linear principal component analysis (NLPCA) and deep learning auto-rectification.
2. The stereoscopic vision system of claim 1 , the processor further comprising:
an image data gathering module; and
a tracking data gathering module.
3. The stereoscopic vision system of claim 2 , the processor further comprising:
a stereo calibration module; and
a spatial calibration module.
4. The stereoscopic vision system of claim 2 , the processor further comprising an iSV reconstruction module.
5. The stereoscopic vision system of claim 2 , further comprising the surgical navigation system.
6. The stereoscopic vision system of claim 5 , further comprising an external monitor display system or heads-up display within the surgeon's field of view.
7. A method of performing surgery using a handheld stereovision system, the method comprising:
obtaining a handheld stereovision (HHS) device, the HHS device including at least two cameras and a surgical tracker;
gathering image information from the two or more cameras and location data from the surgical tracker at the same time;
reconstructing intraoperative stereovision (iSV) images;
registering preoperative CT (pCT) images with the iSV images to generate updated CT (uCT) images, wherein registering pCT images with the iSV images comprises non-linear principal component analysis (NLPCA) and deep learning auto-rectification;
uploading the uCT images to a surgical navigation system; and
performing a surgery in a surgery treatment area using the uCT images with the surgical navigation system.
8. The method of claim 7 , further comprising:
calibrating stereo images; and
calibrating spatial images.
9. The method of claim 7 , wherein the surgery treatment area is a lumbar region of the vertebra.
10. The method of claim 7 , further comprising moving the HHS by hand over the surgery treatment area to increase a size of an area being analyzed for surgery.
11. The method of claim 7 , wherein registering pCT images with the iSV images comprises:
identifying longitudinal and cross-sectional axes of pCT spine surfaces via non-linear principal component analysis (NLPCA); and
flattening the identified surfaces by representing the surfaces as evenly sampled surface meshes and mapping to a set of equal-sized planar Cartesian grids.
12. The method of claim 7 , wherein registering pCT images with the iSV images comprises constrained correspondence mapping.
13. An apparatus for using data from a handheld stereoscopic (HHS) device to assist with surgery comprising:
a computer system;
computer code in the computer system, the computer code comprising:
a method for gathering image information from two or more cameras on the HHS;
a method for gathering location information from a tracker on the HHS;
a method for reconstructing intraoperative stereovision (iSV) images;
a method for registering preoperative CT (pCT) images with the iSV images to generate updated CT (uCT) images, wherein registering pCT images with the iSV images comprises non-linear principal component analysis (NLPCA) and deep learning auto-rectification; and
a method for uploading uCT images to a surgical navigation system.
14. The apparatus of claim 13 , wherein the computer code further comprises a method for displaying the uCT images to a surgeon during surgery to improve surgical outcomes.
15. The apparatus of claim 13 , wherein the computer code further comprises:
a method of calibrating stereo images; and
a method of calibrating spatial images.