Mixed reality imaging apparatus and surgical suite
System and method for use of mixed and augmented reality Imaging surgical suite and an augmented reality device are provided. A 2D/3D virtual grid having a head-up display with augmented reality is provided. The head-up display can display a 2D radiographic image or 3D volumetric representation or shape model of image, and can further display an augmented reality grid/indicator which can be depicted as a grid, implant, instrument or bone avatar/figure representing shapes, orientations, and positions relative to an anatomical image or model. The augmented reality indicator can be displayed directly in a surgeon's field of view to provide a live, intra-operative situational environment and navigational guidance for the surgeon. The head-up display can show avatar grids/indicators that are outside of the surgeon's field of view within a patient's body.
1. A computerized method of enhancing an intraoperative image directly displayed to a user as a graphical user interface, comprising the steps of:
providing a processor, a memory communicably coupled to the processor, a display communicably coupled to the processor and an image source of an anatomical image, communicably coupled to the processor;
receiving the intraoperative image from the image source, wherein the intraoperative image is selected from the group consisting of: an ultrasound, a computed tomography and a magnetic resonance image of a patient during a procedure;
selecting at least one anatomical point within the intraoperative image using the processor;
generating an indicator, wherein said indicator is a grid corresponding to surgical variables associated with the at least one anatomical point using the processor;
creating an enhanced image by combining the intraoperative image with the indicator associated with the at least one anatomical point using the processor; and
displaying the enhanced image on the display as a graphical user interface, wherein said image is enhanced with augmented reality data.
2. The method of claim 1 wherein the display is a heads-up display.
3. The method of claim 1 wherein the enhanced image is selected from the group consisting of: an augmented reality image and an augmented reality volumetric shape model of the intraoperative image.
4. The method of claim 1 further comprising the step of adding an avatar as an indicator, wherein the avatar represents an object selected from the group consisting of: an implant, an instrument and a bone.
5. The method of claim 1 wherein creating an enhanced image comprises spatially overlying the at least one anatomical point of the indicator and the intraoperative image.
6. The method of claim 5 further comprising distorting the indicator image.
7. The method of claim 5 further comprising distorting the intraoperative image.
8. An intraoperative surgical system comprising: a non-transitory computer-readable storage medium encoded with computer-readable instructions which form the application software and a processor to process the instructions to:
receive a surgical image from an image source, wherein the surgical image is selected from the group consisting of: an ultrasound, a computed tomography and a magnetic resonance image of a patient during a surgical procedure;
receive a selection of at least one anatomical point within an intraoperative surgical image using the processor;
generate an indicator wherein said indicator is a grid corresponding to surgical variables associated with the at least one anatomical point using the processor;
create an enhanced image by combining the intraoperative surgical image with the indicator associated with the at least one anatomical point using the processor and a device configured to display the enhanced image to a user as a graphical user interface.
9. The intraoperative surgical system of claim 8 , wherein the device is a heads-up display comprised of: a mixed reality visualization screen and a controller.
10. The intraoperative surgical system of claim 8 , wherein the computer readable storage medium encoded with computer-readable instructions, which form the application software, cause a computer to function as the mixed-reality presentation system.
11. The intraoperative surgical system of claim 9 , wherein the enhanced image is selected from the group consisting of: an augmented reality radiographic image and an augmented reality volumetric shape model of the surgical image.
12. The intraoperative surgical system of claim 9 wherein the enhanced image is configured to provide a surgically integrated visual guidance system, wherein said display is selected from the group consisting of: an augmented reality grid, a holographic grid, and an avatar.
13. The intraoperative surgical system of claim 11 , wherein the indicator is an avatar.
14. The intraoperative surgical system of claim 8 , further comprising
a preoperative sub-artificial reality space; and
an intraoperative sub-artificial reality space and wherein the intraoperative artificial reality sub-space is configured to use an anatomical image.
15. The system of claim 14 further comprising a resource sub-space.
16. The system of claim 8 wherein said surgical procedure is selected from the group consisting of: joint replacements, trauma fracture reductions and deformity correction, implant alignment and implant placement.