Surgical navigation with stereovision and associated methods
A surgical guidance system has two cameras to provide stereo image stream of a surgical field; and a stereo viewer. The system has a 3D surface extraction module that generates a first 3D model of the surgical field from the stereo image streams; a registration module for co-registering annotating data with the first 3D model; and a stereo image enhancer for graphically overlaying at least part of the annotating data onto the stereo image stream to form an enhanced stereo image stream for display, where the enhanced stereo stream enhances a surgeon's perception of the surgical field. The registration module has an alignment refiner to adjust registration of the annotating data with the 3D model based upon matching of features within the 3D model and features within the annotating data; and in an embodiment, a deformation modeler to deform the annotating data based upon a determined tissue deformation.
1. A system for generating a photographic image of a surgical field, comprising:
a stereo image capture device for capturing a stereo image stream from two optical cameras imaging the surgical field during a surgical procedure;
a stereo viewer for displaying the stereo image stream to a surgeon;
a stereo image to 3D surface model extraction module, implemented as machine readable instructions stored in memory of a computer, adapted to, when the instructions are executed by a processor of the computer, generate a first 3D model of the surgical field from the stereo image stream;
a registration module, implemented as machine readable instructions stored in the memory, adapted to, when the instructions are executed by a processor of the computer, co-register annotating data with the first 3D model; and
a stereo image enhancer, implemented as machine readable instructions stored in the memory, adapted to, when the instructions are executed by a processor of the computer, graphically overlay at least part of the annotating data onto the stereo image stream to form an enhanced stereo image stream for display by the stereo viewer, wherein the enhanced stereo stream enhances the surgeon's perception of the surgical field.
2. The system of claim 1 , wherein the registration module aligns the annotating data with the first 3D model.
3. The system of claim 1 , further comprising:
a 3D model generator for generating a second 3D model from the annotating data, wherein the registration module co-registers the second 3D model with the first 3D model; and
an image generator configured to generate a graphical image of the second 3D model based upon the surgical field.
4. The system of claim 3 , wherein the annotating data comprises one of a preoperative, an intraoperative and a postoperative radiological study selected from the group consisting of magnetic resonance imaging (MRI), functional magnetic resonance imaging (fIVIRI), x-ray computed tomography (CT), Single-photon emission computed tomography (SPECT), and positron emission tomography (PET).
5. The system of claim 4 , the image generator configured to generate a cross sectional view of the second 3D model based upon an indicated surgical plane within the surgical field.
6. A system for generating a photographic image of a surgical field, comprising:
a stereo image capture device for capturing a stereo image stream from two cameras imaging the surgical field during a surgical procedure;
a stereo viewer for displaying the stereo image stream to a surgeon;
a stereo image to 3D surface extraction module, implemented as machine readable instructions stored in memory of a computer, configured to, when the instructions are executed by a processor of the computer, generate a first 3D model of the surgical field from the stereo image stream;
a registration module, implemented as machine readable instructions stored in the memory, configured to, when the instructions are executed by the processor, co-register annotating data with the first 3D model;
a stereo image enhancer, implemented as machine readable instructions stored in the memory, configured to, when the instructions are executed by the processor, graphically overlay at least part of the annotating data onto the stereo image stream to form an enhanced stereo image stream for display by the stereo viewer, wherein the enhanced stereo stream enhances the surgeon's perception of the surgical field;
a 3D model generator for generating a second 3D model from the annotating data, wherein the registration module co-registers the second 3D model with the first 3D model; and
an image generator configured to generate a graphical image of the second 3D model based upon the surgical field;
wherein the annotating data further comprises physiological study data selected from the group including EEG, evoked potentials, and magnetoencephalography (MEG).
7. The system of claim 1 , the registration module further comprising an alignment refiner configured to adjust the registration of the annotating data with the 3D model based upon matching of features within the 3D model and features within the annotating data.
8. The system of claim 1 , the registration module further comprising a deformation modeler configured to deform the annotating data based upon a determined tissue deformation within the surgical field.
9. The system of claim 8 , the tissue deformation resulting from one or more of skull removal, retraction, and incision.
10. The system of claim 4 , further comprising:
a first controllable filter coupled with a first of the two cameras to filter light entering the first of the two cameras from the surgical field;
a first controllable light source for illuminating the surgical field with light of a determined fluorescent stimulus wavelength; and
a fluorescence depth processor, implemented as machine readable instructions stored in the memory, adapted to, when the instructions are executed by the processor, generate a 3D fluorescence model;
wherein the stereo image enhancer graphically overlays at least a portion of the 3D fluorescence model onto the enhanced stereo image stream.
11. The system of claim 10 , further comprising:
a second controllable filter coupled with a second of the two cameras to filter light entering the second of the two cameras from the surgical field; and
a hyperspectral image processor, implemented as machine readable instructions stored in the memory, adapted to, when the instructions are executed by the processor, control the first and second filters to capture a stereo hyperspectral image stream and generate a 3D hyperspectral model;
wherein the stereo image enhancer graphically overlays at least a portion of the 3D hyperspectral model onto the enhanced stereo image stream.
12. The system of claim 1 , further comprising:
a first and a second hyperspectral imaging camera for capturing a stereo hyperspectral image stream of the surgical field; and
a hyperspectral image processor, implemented as machine readable instructions stored in the memory, adapted to, when the instructions are executed by the processor, generate a 3D hyperspectral model from the stereo hyperspectral image stream;
wherein the stereo image enhancer graphically overlays at least a portion of the 3D hyperspectral model onto the enhanced stereo image stream.
13. The system of claim 3 , further configured to review a recorded surgical procedure that is co-registered with post operatively acquired imaging and/or physiological data.
14. A method for surgical navigation with stereo vision, comprising:
capturing a stereo image pair of a surgical field during an operation on the surgical field;
generating a 3D surface model from the stereo image pair;
registering annotating data with the 3D surface model, said registering including adjusting the annotating data based upon matched features of the 3D surface model and features of the annotating data, said adjusting including generating a mechanical model of the tissue within the surgical field to determine an adjustment;
generating an enhanced stereo image based upon the stereo image pair and the annotating data as registered with the 3D surface model;
wherein the annotating data is graphically overlaid onto the stereo image pair as registered with the 3D surface model.
15. The method of claim 14 , the step of registering further comprising:
generating a 3D model based upon the annotating data; and
registering the 3D model to the 3D surface model.
16. The method of claim 14 , the step of adjusting comprising one or more of shifting, rotating, warping, and scaling the annotating data.
17. A method for surgical navigation with stereo vision, comprising:
capturing a stereo image pair of a surgical field during an operation on the surgical field;
generating a 3D surface model from the stereo image pair;
registering annotating data with the 3D surface model;
generating an enhanced stereo image based upon the stereo image pair and the annotating data;
wherein the annotating data is graphically overlaid onto the stereo image pair, and
wherein the annotating data further comprises a physiological study selected from the group including EEG, evoked potentials, and magnetoencephalography (MEG).
18. The method of claim 17 , further comprising recording the enhanced stereo image pair.
19. The method of claim 14 , wherein the annotating data comprises one of a preoperative, an intraoperative and a postoperative radiological study selected from the group consisting of magnetic resonance imaging (MRI), functional magnetic resonance imaging (fMRI), x-ray computed tomography (CT), Single-photon emission computed tomography (SPECT), and positron emission tomography (PET).
20. The method of claim 17 , further comprising recording the stereo image pair.
21. The method for surgical navigation of claim 17 , wherein the annotating data is derived from magnetic resonance imaging (MRI) or computed tomography (CT) image stacks.
22. The method for surgical navigation of claim 17 , wherein the annotating data comprises physiological data.
23. The system of claim 1 wherein the annotating data comprises physiological data.