Mixed reality imaging system, apparatus and surgical suite
System and method for use of mixed and augmented reality Imaging surgical suite and an augmented reality device are provided. A 2D/3D virtual grid having a head-up display with augmented reality is provided. The head-up display can display a 2D radiographic image or 3D volumetric representation or shape model of image, and can further display an augmented reality grid/indicator which can be depicted as a grid, implant, instrument or bone avatar/figure representing shapes, orientations, and positions relative to an anatomical image or model. The augmented reality indicator can be displayed directly in a surgeon's field of view to provide a live, intra-operative situational environment and navigational guidance for the surgeon. The head-up display can show avatar grids/indicators that are outside of the surgeon's field of view within a patient's body.
1. A computerized method of enhancing an intraoperative surgical image directly displayed to a user as a graphical user interface, comprising the steps of:
providing a processor, a memory communicably coupled to the processor, a display communicably coupled to the processor and an image source of an anatomical image, communicably coupled to the processor;
receiving a surgical image from the image source; wherein the surgical image is an X-ray image, of a patient during a surgical procedure;
selecting at least one anatomical point within the surgical image using the processor;
generating an indicator, wherein said indicator is a grid corresponding to surgical variables, associated with the at least one anatomical point using the processor;
creating an enhanced image by combining the surgical image with the indicator associated with the at least one anatomical point using the processor; and
displaying the enhanced image on the display as a graphical user interface, wherein said image is enhanced with augmented reality data.
2. The method of claim 1 wherein the display is a heads-up display.
3. The method of claim 1 wherein the enhanced image is selected from the group consisting of: an augmented reality radiographic image and an augmented reality volumetric shape model of the surgical image.
4. The method of claim 1 further comprising the step of adding an avatar as an indicator.
5. The method of claim 4 wherein the avatar represents an object selected from the group consisting of: an implant, an instrument and a bone.
6. The method of claim 1 wherein creating an enhanced image comprises spatially overlying the at least one anatomical point of the indicator and the surgical image.
7. The method of claim 6 further comprising distorting an indicator image.
8. The method of claim 6 further comprising distorting the surgical image.
9. An intraoperative surgical system comprising: a non-transitory computer-readable storage medium encoded with computer-readable instructions which form the application software and a processor to process the instructions to:
receive a surgical image from an image source; wherein the surgical image is an X-ray image, of a patient during a surgical procedure;
receive a selection of at least one anatomical point within an intraoperative surgical image using the processor;
generate an indicator wherein said indicator is a grid corresponding to surgical variables associated with the at least one anatomical point using the processor;
create an enhanced image by combining the intraoperative surgical image with the indicator associated with the at least one anatomical point using the processor and a device configured to display the enhanced image to a user as a graphical user interface.
10. The intraoperative surgical system of claim 9 , wherein the device is a heads up display comprised of: a mixed reality visualization screen and a controller.
11. The intraoperative surgical system of claim 10 , wherein the enhanced image is selected from the group consisting of: an augmented reality radiographic image and an augmented reality volumetric shape model of the surgical image.
12. The intraoperative surgical system of claim 11 , wherein the indicator is an avatar.
13. The intraoperative surgical system of claim 10 wherein the enhanced image is configured to provide a surgically integrated visual guidance system, wherein said display is selected from the group consisting of: an augmented reality grid, a holographic grid, and an avatar.
14. The intraoperative surgical system of claim 9 wherein the computer readable storage medium encoded with computer-readable instructions which form the application software, cause a computer to function as the mixed-reality presentation system.
15. The intraoperative surgical system of claim 9 , further comprising
a preoperative sub-artificial reality space; and
an intraoperative sub-artificial reality space.
16. The system of claim 15 further comprising a resource sub-space.
17. The system of claim 15 wherein the intraoperative artificial reality sub-space is configured to use an anatomical image.
18. The system of claim 9 wherein said surgical procedure is selected from the group consisting of: joint replacements, trauma fracture reductions and deformity correction, implant alignment and implant placement.