Methods and devices for optical aberration correction
Near-to-eye displays within head mounted devices offer both users with and without visual impairments enhanced visual experiences either by improving or augmenting their visual perception. Unless the user directly views the display without intermediate optical elements then the designer must consider chromatic as well as other aberrations. Within the prior art the optical train is either complex through additional corrective elements adding to weight, cost, and size or through image processing. However, real time applications with mobile users require low latency to avoid physical side effects. Accordingly, it would be beneficial to provide near-to-eye displays mitigating these distortions and chromatic aberrations through pre-distortion based electronic processing techniques in conjunction with design optimization of the optical train with low weight, low volume, low complexity, and low cost. Further, it would be beneficial to exploit consumer grade low cost graphics processing units rather than application specific circuits.
1. A method comprising:
receiving image data for presentation to a user on a near-to-eye (N2I) display wherein an optical train is disposed between the user's eye and the N2I display;
processing the received image data in dependence upon a static vertex buffer with an image processing pipeline, the static vertex buffer defining for each point within a grid representing an effective visual frame to be presented to the user a plurality of pixel locations within an image to be displayed to the user; wherein
each pixel location of the plurality of pixel locations is associated with a predetermined colour pixel within the image; and
each pixel location of the plurality of pixel locations when subjected to the chromatic aberration of a prism lens which couples the N2I display to a user's eye approximately aligns to a point within the grid;
storing the processed output from the image processing pipeline within a frame buffer; and
displaying the processed output to the user on the N2I display.
2. The method according to claim 1 , wherein
the optical train comprises a prism lens, wherein the N2I display is coupled to a first facet of the prism lens wherein the image from the N2I is reflected from a second facet of the prism lens to a third facet of the prism lens and therein to user's eye.
3. The method according to claim 1 , wherein
the processing is performed using a graphical processing unit (GPU) comprising at least a vertex shader and fragment shader, wherein
colour-dependent spatial transforms and spatially-dependent amplitude transforms are implemented by the fragment shader and not the vertex shader.
4. The method according to claim 1 , wherein
the static vertex buffer comprises for each vertex in a defined vertex array a texture attribute for each pixel colour within the N2I display together with the vertex's associated coordinates within the N2I display.
5. The method according to claim 1 , wherein
the static vertex buffer comprises for each vertex in a defined vertex array a vertex attribute for each pixel colour within the N2I display established in dependence upon a chief ray angle of a pixel within an acquiring image sensor associated with the vertex in the defined vertex array.
6. The method according to claim 1 , wherein
the static vertex buffer defines which display pixels within the N2I display should be activated for each image pixel within the image to be displayed; wherein
there is a one-to-one mapping of image pixels to display pixels.
7. The method according to claim 6 , wherein
the texture attributes at each vertex within the static vertex buffer define corrections for at least one of optical aberrations and chromatic aberrations within at least one of the optical train and the acquiring image sensor.
8. The method according to claim 1 , wherein
the processing is performed using a graphical processing unit (GPU) such process comprising at least a vertex shader and a fragment shader, wherein
the received image data is coupled to the fragment shader.
9. The method according to claim 1 , wherein the static vertex buffer is a static texture vertex array.
10. The method according to claim 1 , wherein
processing the received image data in dependence upon the static vertex buffer also comprises processing the received image data in dependence upon a texture object.
11. The method according to claim 1 , wherein
the vertices comprising the static vertex buffer each define a triangular tesselation of a plane.
12. Computer executable code for execution by a microprocessor stored upon a non-volatile, non-transient memory, the computer executable code comprising instructions relating to a method comprising the steps:
receiving image data for presentation to a user on a near-to-eye (N2I) display wherein an optical train is disposed between the user's eye and the N2I display;
processing the received image data in dependence upon a static vertex buffer with an image processing pipeline, the static vertex buffer defining for each point within a grid representing an effective visual frame to be presented to the user a plurality of pixel locations within an image to be displayed to the user; wherein
each pixel location of the plurality of pixel locations is associated with a predetermined colour pixel within the image; and
each pixel location of the plurality of pixel locations when subjected to the chromatic aberration of a prism lens which couples the N2I display to a user's eye approximately aligns to a point within the grid;
storing the processed output from the image processing pipeline within a frame buffer; and
displaying the processed output to the user on the N2I display.
13. The computer executable code according to claim 12 , wherein
the optical train comprises a prism lens, wherein the N2I display is coupled to a first facet of the prism lens wherein the image from the N2I is reflected from a second facet of the prism lens to a third facet of the prism lens and therein to user's eye.
14. The computer executable code according to claim 12 , wherein
the processing is performed using an application programming interface for rendering vector graphics in conjunction with a graphical processing unit (GPU) comprising at least a vertex shader and a fragment shader, wherein
the fragment shader implements colour-dependent spatial transforms and spatially-dependent amplitude transforms.
15. The computer executable code according to claim 12 , wherein
the static vertex buffer comprises for each vertex in a defined vertex array a texture attribute for each pixel colour within the N2I display together with the vertex's associated coordinates within the N2I display.
16. The computer executable code according to claim 12 , wherein
the static vertex buffer comprises for each vertex in a defined vertex array a vertex attribute for each pixel colour within the N2I display established in dependence upon a chief ray angle of a pixel within an acquiring image sensor associated with the vertex in the defined vertex array.
17. The computer executable code according to claim 12 , wherein
the static vertex buffer defines which display pixels within the N2I display should be activated for each image pixel within the image to be displayed; wherein
there is a one-to-one mapping of image pixels to display pixels.
18. The computer executable code according to claim 17 , wherein
the texture attributes at each vertex within the static vertex buffer defines corrections for at least one of optical aberrations and chromatic aberrations within at least one of the optical train and the acquiring image sensor.
19. The computer executable code according to claim 12 , wherein
the processing is performed using an application programming interface for rendering vector graphics process in conjunction with a graphical processing unit (GPU) comprising at least a vertex shader and fragment shader, wherein
the received image data is coupled to the fragment shader.