Systems for facilitating augmented reality-assisted medical procedures
Disclosed herein is a near eye assembly having a retaining structure (e.g., a spectacle frame or head up display mounted on a helmet) that is configured to be positioned in proximity to the eye of a user of the assembly. An optical combiner may be mounted on the retaining structure in front of the user eye. The optical combiner may at least partially transmit elements of a scene in front of the assembly through the combiner. In addition, the optical combiner may receive a visible radiation transmission derived from a scene, and/or a visual transmission such as a presentation of data or a marker, and redirect the transmission back to the user's eye.
1. A system for facilitating an augmented reality assisted medical procedure on a patient, the system comprising:
a region of interest marker comprising marker elements configured as retro-reflectors that selectively reflect infra-red radiation, the region of interest marker configured to be positioned at a region of interest of the patient associated with the medical procedure;
one or more device identifying elements configured to be attached to a surgical device; and
an augmented reality assembly comprising a retaining structure configured to be mounted on a head of a medical professional, wherein the retaining structure comprises:
one or more processors configured to access a database having one or more stored images;
a camera configured to operate in an infra-red spectrum and to capture images of the one or more device identifying elements and the marker elements of the region of interest marker;
two optical combiners mounted on the retaining structure, each one of the two optical combiners and positioned so as to be in front of a respective one of the eyes of the medical professional,
wherein the two optical combiners are configured to be rotated about a horizontal axis such that they can tilt downwards at an angle of up to 40 degrees from horizontal,
wherein each optical combiner comprises a pixelated variable transparency screen that is configured to at least partially transmit elements of a scene through the optical combiner so that a portion of the region of interest is directly visible through the optical combiner; and
two micro-projectors mounted on the retaining structure, each micro-projector located and oriented so as to be able to project selected images from the database onto a respective optical combiner,
wherein the one or more processors are configured to control opacity of each of the pixels of the pixelated variable transparency screen, and
wherein the one or more processors are further configured to align the projection of the selected images with the region of interest based on the captured images of the one or more device identifying elements and the marker elements of the region of interest marker.
2. The system of claim 1 , wherein the retaining structure comprises a spectacle frame.
3. The system of claim 1 , wherein the retaining structure comprises a helmet.
4. The system of claim 1 , further comprising two lenses, each of the two lenses being positioned between a respective one of the two optical combiners and a respective one of the eyes of the medical professional, wherein each of the two lenses redirects light from a respective optical combiner so that the projection of the selected images appears to be closer than infinity to the eyes of the medical professional.
5. The system of claim 4 , wherein the selected images are adapted to be in focus at approximately 50 cm.
6. The system of claim 1 , wherein the screen is formed of an array of liquid crystal pixels.
7. The system of claim 6 , wherein the array is a rectangular array.
8. The system of claim 1 , wherein the one or more processors are configured to render at least a section of the screen at least partially opaque.
9. The system of claim 8 , wherein the section is a circular section.
10. The system of claim 1 , wherein any misalignment between the projection of the selected images and the region of interest is less than 2 cm.
11. The system of claim 1 , wherein the optical combiners are of a waveguide type.
12. A system for facilitating an augmented reality assisted medical procedure on a patient, the system comprising:
a region of interest marker comprising marker elements configured as retro-reflectors that selectively reflect infra-red radiation, the region of interest marker configured to be positioned at a region of interest of the patient associated with the medical procedure;
one or more device identifying elements configured to be attached to a medical device; and
an augmented reality assembly comprising a retaining structure configured to be mounted on a head of a medical professional, wherein the retaining structure comprises:
one or more processors configured to access a database having one or more stored images;
a camera configured to operate in an infra-red spectrum and to capture images of the one or more device identifying elements and the marker elements of the region of interest marker;
two optical combiners mounted on the retaining structure, each one of the two optical combiners and positioned so as to be in front of a respective one of the eyes of the medical professional,
wherein the two optical combiners are configured to be rotated about a horizontal axis such that they can tilt downwards at an angle of up to 40 degrees from horizontal,
wherein each optical combiner comprises a variable transparency screen that is configured to at least partially transmit elements of a scene through the optical combiner so that a portion of the region of interest is directly visible through the optical combiner; and
two micro-projectors mounted on the retaining structure, each micro-projector located and oriented so as to be able to project selected images from the database onto a respective optical combiner,
wherein the one or more processors are configured to control opacity of the variable transparency screen, and
wherein the one or more processors are further configured to align the projection of the selected images with the region of interest based on the captured images of the one or more device identifying elements and the marker elements of the region of interest marker.
13. The system of claim 12 , wherein the retaining structure comprises a spectacle frame.
14. The system of claim 12 , wherein the retaining structure comprises a helmet.
15. The system of claim 12 , further comprising two lenses, each of the two lenses being positioned between a respective one of the two optical combiners and a respective one of the eyes of the medical professional, wherein each of the two lenses redirects light from a respective optical combiner so that the projection of the selected images appears to be closer than infinity to the eyes of the medical professional.
16. The system of claim 15 , wherein the selected images are adapted to be in focus at approximately 50 cm.
17. The system of claim 12 , wherein the screen is pixelated.
18. The system of claim 17 , wherein the screen is formed of an array of liquid crystal pixels.
19. The system of claim 12 , wherein each optical combiner comprises a waveguide configured to transmit light coming from a respective one of the two micro-projectors into the eyes of the medical professional so as to display the selected images from the database as an augmented reality display.