Surgical navigation with stereovision and associated methods
A surgical guidance system has two cameras to provide stereo image stream of a surgical field; and a stereo viewer. The system has a 3D surface extraction module that generates a first 3D model of the surgical field from the stereo image streams; a registration module for co-registering annotating data with the first 3D model; and a stereo image enhancer for graphically overlaying at least part of the annotating data onto the stereo image stream to form an enhanced stereo image stream for display, where the enhanced stereo stream enhances a surgeon's perception of the surgical field. The registration module has an alignment refiner to adjust registration of the annotating data with the 3D model based upon matching of features within the 3D model and features within the annotating data; and in an embodiment, a deformation modeler to deform the annotating data based upon a determined tissue deformation.
1. A system for generating a photographic image of tissue, comprising:
a stereo image capture device for capturing a stereo image stream from two optical cameras imaging the tissue during a surgical procedure;
a viewer for displaying the stereo image stream to a surgeon;
a stereo image to 3D surface model extraction module, implemented as machine readable instructions stored in memory of a computer, adapted to, when the instructions are executed by a processor of the computer, generate a first 3D model of the tissue from the stereo image stream;
a registration module, implemented as machine readable instructions stored in the memory, adapted to, when the instructions are executed by a processor of the computer, co-register annotating data with the first 3D model; and
a stereo image enhancer, implemented as machine readable instructions stored in the memory, adapted to, when the instructions are executed by a processor of the computer, graphically overlay at least part of the annotating data and at least some live data onto the stereo image stream to form an enhanced stereo image stream for display by the viewer, wherein the enhanced stereo stream,
wherein the live data comprises fluorescent imaging data or at least one physiological parameter selected from hemoglobin oxygenation levels determined from imaging absorbance parameters of the tissue at each of several wavelengths of light and hemoglobin concentration.
2. The system of claim 1 wherein the registration module automatically determines a spatial relationship between the 3D model and features of patient bone structure from preoperative scans and determines if there is an inconsistency, and if an inconsistency is found generates a warning.
3. The system of claim 1 wherein the annotating data comprises images representing planned surgical procedures.
4. The system of claim 3 wherein the images representing planned surgical procedures are rendered images from a 3D model adapted to represent the tissue as deformed during surgery.
5. The system of claim 3 wherein the annotating data comprises images representing implantable devices.
6. The system of claim 5 further comprising a recognition module adapted to recognize an implantable device.
7. The system of claim 6 wherein the registration of annotating data comprises adjustment for tissue deformation due to presence of the implantable device.
8. The system of claim 7 wherein the implantable device is selected from the group consisting of a of a deep-brain probe, deep-brain stimulator, and a subdural electrode array.
9. The system of claim 1 wherein the live data comprises fluorescence intensity.
10. The system of claim 1 wherein the live data comprises depth of fluorophore concentrations in the tissue.
11. The system of claim 10 wherein the depth of fluorophore concentrations is determined by observing fluorescent emissions from fluorophore in the tissue at a plurality of wavelengths.
12. The system of claim 1 where the live data comprises hemoglobin oxygenation levels.
13. The system of claim 12 where the live data comprises hemoglobin concentrations.
14. A method for surgical navigation with stereo vision, comprising:
capturing a stereo image pair of a surgical field during an operation on the surgical field;
generating a 3D surface model from the stereo image pair;
registering annotating data with the 3D surface model, said registering including adjusting the annotating data based upon matched features of the 3D surface model and features of the annotating data, said adjusting including generating a mechanical model of tissue within the surgical field;
generating an alarm if inconsistency between bony features of the annotating data are inconsistent with features extracted from the 3D surface model; and
generating an enhanced stereo image based upon the stereo image pair, live data, and the annotating data as registered with the 3D surface model;
wherein the annotating data is graphically overlaid onto the stereo image pair as registered with the 3D surface.
15. The method of claim 14 wherein the live data comprises hemoglobin oxygenation levels determined from imaging absorbance parameters of the tissue at each of several wavelengths of light.
16. The method for surgical navigation with stereo vision of claim 14 wherein the live data comprises fluorescence intensity.
17. The method for surgical navigation with stereo vision of claim 14 wherein the live data comprises depth of fluorophore concentrations in the tissue.
18. The method for surgical navigation with stereo vision of claim 17 wherein the depth of fluorophore concentrations is determined by observing fluorescent emissions from fluorophore in the tissue at a plurality of wavelengths.
19. The method for surgical navigation of claim 14 wherein the live data comprises hemoglobin oxygenation.