Augmented and extended reality glasses for use in surgery visualization and telesurgery
An augmented reality and extended reality surgical system. The system may comprise a wearable device, such as a head mounted display or glasses, that provides the user with virtual reality, augmented reality, and/or mixed reality for surgery visualization. This may allow the user to access 2D or 3D imaging, magnification, virtual visualization, six-degrees of freedom (6DoF) image management, and/or other images while still viewing real reality and thus maintaining a presence in the operating room.
1. An AXR surgical system comprising:
a wearable device comprising:
one or more micro-displays;
one or more lenses, where the micro-displays are capable of projecting images onto the lenses;
a head-tracking subsystem; and
an eye-tracking subsystem; and
a central processing unit in communication with and capable of controlling the micro-displays, lenses, head-tracking subsystem, and eye-tracking subsystem;
where the system is capable of displaying images on the lenses with a position based on a user's head position as tracked by the head-tracking subsystem and the user's eye position as tracked by the eye-tracking subsystem, while allowing the user to see through the lenses where the images are not being projected; and
where the lenses comprise:
a reflective layer; and
a layer of cholesteric liquid crystal comprising a plurality of pixels where each pixel is capable of independently becoming opaque,
where the system is capable of selectively making pixels opaque only where images are projected by the micro-displays, while any pixels located where the images are not projected remain see-through.
2. The AXR surgical system of claim 1 where the camera lenses are capable of capturing a wider field of vision than the micro-displays are capable of projecting a reduced field of vision depending on the user's eye position.
3. The AXR surgical system of claim 1 where the wearable device further comprises one or more forward-facing cameras and where the images projected by the micro-displays are at least partially images obtained from the forward-facing cameras.
4. The AXR surgical system of claim 1 further comprising one or more microphones in communication with the central processing unit, where the system is capable of being controlled via voice input via the microphones, input from the eye-tracking subsystem, or a combination of voice input via the microphones and input from the eye-tracking subsystem.
5. The AXR surgical system of claim 4 where the microphones have noise cancelling features capable of reducing ambient noise.
6. The AXR surgical system of claim 1 where the wearable device further comprises one or more batteries and a hot-swap feature.
7. The AXR surgical system of claim 6 where the wearable device further comprises a remote communication device such that the wearable device is wireless.
8. The AXR surgical system of claim 1 where the system is in communication with one or more second systems such that one or more remote users can view the images from any system on the one or more second systems and communicate with the user and other remote users.
9. The AXR surgical system of claim 1 where the images projected by the micro-displays are from preoperative imaging and where the system is capable of aligning the images with a patient.
10. The AXR surgical system of claim 1 where the system is in communication with one or more second systems such that one or more remote users can view images from an originating system on the one or more second systems in the same or different perspective as exists on the originating system and communicate with the user and other remote users.
11. The AXR surgical system of claim 1 where the system is in communication with one or more second systems such that one or more remote users can view the images from a system on the one or more second systems and analyze the images with AI technologies the result of which would be visible to all users.
12. The AXR surgical system of claim 1 further comprising at least one collimator located between the micro-displays and the reflective layer such that the at least one collimator is capable of concentrating rays from the micro-displays in an eye box while utilizing less resolution in a periphery.
13. The AXR surgical system of claim 1 where the wearable device further comprises one or more at least partially downward-facing cameras and where the images projected by the micro-displays are at least partially images obtained from the at least partially downward-facing cameras.
14. The AXR surgical system of claim 1 further comprising a remote camera system in wireless communication with the wearable device, where the images come from the remote camera system.
15. The AXR surgical system of claim 14 where the remote camera system is mounted on a six-axis cobotic arm.
16. The AXR surgical system of claim 15 where the cobotic arm is in communication with the system such that the cobotic arm is controlled by the user.
17. The AXR surgical system of claim 1 where the system is capable of centering an image in a field of view of the user and subsequently repositioning the image based on the user's eye position as tracked by the eye-tracking subsystem.
18. The AXR surgical system of claim 1 where the wearable device is wireless and is synced with one or more wired display systems such that the system is capable of presenting a simultaneous view on the wearable device and on the one or more wired display systems.