Wearable image manipulation and control system with high resolution micro-displays and dynamic opacity augmentation in augmented reality glasses
A mixed reality display comprising: at least one lens, where the at least one lens has a reflective element, the at least one lens comprising a plurality of pixels; at least one display capable of projecting one or more images onto at least a portion of the at least one lens; and a dynamic opacity system, where the dynamic opacity system is capable of making at least one pixel opaque in the portion of the at least one lens onto which the one or more images are projected, while any portion of the at least one lens onto which no image is projected remains see-through.
1. An augmented reality system comprising:
a wearable device configured to be worn by a patient including:
a lens subsystem including:
a transparent layer with a first surface and an opposed second surface;
a reflective coating applied to the first surface of the transparent layer; and
an alpha matte layer abutting the second surface of the transparent layer, the alpha matte layer including a cholesteric liquid crystal layer including a plurality of pixels of cholesteric liquid crystal, where each pixel is capable of independently becoming opaque or transparent; and
one or more micro-displays capable of projecting images onto the lens subsystem reflective coating;
a retinal map associated with the patient, where the retinal map comprises at least one boundary indicative of an area to be modified within the patients visual perception;
a display controller in communication with the wearable device capable of receiving and storing the retinal map, capable of receiving a streaming video comprising one or more frames from a camera associated with the patient, capable of buffering the one or more frames of the streaming video, capable of identifying a center of each frame and modifying the streaming video by stretching pixels from the center within the area to be modified within the patient's visual perception to a peripheral area outside the boundary, capable of displaying the modified streaming video onto the lens subsystem reflective coating using the one or more micro-displays, and capable of displaying a plurality of pixels opaque in the portion of the alpha matte layer aligning with the portion of the reflective coating onto which the modified streaming video is projected, while pixels aligning with any portion of the reflective coating onto which no image is projected remains transparent.
2. The augmented reality system of claim 1 where the center of each frame is determined by eye-tracking.
3. The augmented reality system of claim 1 where the display controller buffers the streaming video by mapping each pixel and moving each mapped pixel to a new location.
4. A method of operating an augmented reality wearable device including a lens subsystem, one or more micro-displays capable of projecting images onto the lens subsystem, and a display controller including one or more processors coupled to the lens subsystem and the one or more micro-displays, the lens subsystem including a transparent layer with a first surface and an opposed second surface, a reflective coating applied to the first surface of the transparent layer, and an alpha matte layer abutting the second surface of the transparent layer, the alpha matte layer including a cholesteric liquid crystal layer including a plurality of pixels of cholesteric liquid crystal, where each pixel is capable of independently becoming opaque or transparent, the method comprising the one or more processors executing an algorithm including the steps of:
receiving and storing a retinal map associated with a patient, where the retinal map comprises at least one boundary indicative of an area to be modified within the patient's visual perception;
receiving a streaming video comprising one or more frames from a camera associated with the patient;
modifying the streaming video by:
buffering the one or more frames of the streaming video;
identifying a center of each frame; and
stretching pixels from the center within the area to be modified within the patient's visual perception to a peripheral area outside the boundary;
displaying the modified streaming video onto the lens subsystem reflective coating using the one or more micro-displays; and
displaying a plurality of pixels opaque in the portion of the alpha matte layer aligning with the portion of the reflective coating onto which the modified streaming video is projected, while pixels aligning with any portion of the reflective coating onto which no image is projected remains transparent.
5. The method of claim 4 where the center of each frame is determined by eye-tracking.
6. The method of claim 4 where stretching pixels from the center within the area to be modified within the patient's visual perception to a peripheral area outside the boundary comprises mapping each pixel and moving each mapped pixel to a new location.