Methods and devices for optical aberration correction
Near-to-eye displays within head mounted devices offer both users with and without visual impairments enhanced visual experiences either by improving or augmenting their visual perception. Unless the user directly views the display without intermediate optical elements then the designer must consider chromatic as well as other aberrations. Within the prior art the optical train is either complex through additional corrective elements adding to weight, cost, and size or through image processing. However, real time applications with mobile users require low latency to avoid physical side effects. Accordingly, it would be beneficial to provide near-to-eye displays mitigating these distortions and chromatic aberrations through pre-distortion based electronic processing techniques in conjunction with design optimization of the optical train with low weight, low volume, low complexity, and low cost. Further, it would be beneficial to exploit consumer grade low cost graphics processing units rather than application specific circuits.
1. A device comprising:
a near-to-eye (N2I) display for displaying a processed image to an eye of a user;
an optical train disposed between N2I display and the user's eye for coupling an image displayed upon the N2I display to the user's eye;
an image processing pipeline for processing input comprising digital image content relating to an image to be displayed and storing the processed image to be accessed by the N2I display; wherein
the processing of the digital image content to generate the processed image is performed in dependence upon a static vertex buffer with the image processing pipeline, the static vertex buffer defining for each point within a grid representing an effective visual frame to be presented to the user a plurality of pixel locations within an image to be displayed to the user; wherein
each pixel location of the plurality of pixel locations is associated with a predetermined colour pixel within the image; and
each pixel location of the plurality of pixel locations when subjected to the chromatic aberration of the optical train which couples the N2I display to a user's eye approximately aligns to a point within the grid;
the processing is performed using a graphical processing unit (GPU) comprising at least a vertex shader and fragment shader; and
colour-dependent spatial transforms and spatially-dependent amplitude transforms are implemented by the fragment shader and not the vertex shader.
2. The device according to claim 1 , wherein
the optical train comprises at least one of:
a prism lens; and
a prism lens disposed such that the N2I display is coupled to a first facet of the prism lens wherein the image from the N2I is reflected from a second facet of the prism lens to a third facet of the prism lens and therein to user's eye.
3. The device according to claim 1 , wherein
at least one of:
the processed output is stored within a frame buffer; and
the static vertex buffer is a static texture vertex array.
4. The device according to claim 1 , wherein
the image processing pipeline comprises a graphical processing unit (GPU) comprising at least a vertex shader and a fragment shader, wherein
the received image data is coupled to the fragment shader.
5. The device according to claim 1 , wherein
the static vertex buffer comprises for each vertex in a defined vertex array at least one of:
a texture attribute for each pixel colour within the N2I display together with the vertex's associated coordinates within the N2I display; and
a vertex attribute for each pixel colour within the N2I display established in dependence upon a chief ray angle of a pixel within an acquiring image sensor associated with the vertex in the defined vertex array.
6. The device according to claim 1 , wherein
the static vertex buffer defines which display pixels within the N2I display should be activated for each image pixel within the image to be displayed; wherein
there is a one-to-one mapping of image pixels to display pixels.
7. The device according to claim 6 , wherein
the texture attributes at each vertex within the static vertex buffer define corrections for at least one of optical aberrations and chromatic aberrations within at least one of the optical train and the acquiring image sensor.
8. The device according to claim 1 , wherein
processing the received image data in dependence upon the static vertex buffer also comprises processing the received image data in dependence upon a texture object.
9. The device according to claim 1 , wherein
the vertices comprising the static vertex buffer each define a triangular tesselation of a plane.
10. A device comprising:
an image processing pipeline for processing input comprising digital image content relating to an image to be displayed and storing the processed digital image content for access by a near-to-eye (N2I) display forming part of a display system presenting images to a user; wherein
the user views the N2I display via an optical train disposed between N2I display and the user's eye; and
the processing of the digital image content to generate the processed digital image content is performed in dependence upon a static vertex buffer with the image processing pipeline, the static vertex buffer defining for each point within a grid representing an effective visual frame to be presented to the user a plurality of pixel locations within an image to be displayed to the user; wherein
each pixel location of the plurality of pixel locations is associated with a predetermined colour pixel within the image; and
each pixel location of the plurality of pixel locations when subjected to the chromatic aberration of the optical train which couples the N2I display to a user's eye approximately aligns to a point within the grid;
the processing is performed using a graphical processing unit (GPU) comprising at least a vertex shader and fragment shader, and
colour-dependent spatial transforms and spatially-dependent amplitude transforms are implemented by the fragment shader and not the vertex shader.
11. The device according to claim 10 , wherein
the optical train comprises at least one of:
a prism lens; and
a prism lens disposed such that the N2I display is coupled to a first facet of the prism lens wherein the image from the N2I is reflected from a second facet of the prism lens to a third facet of the prism lens and therein to user's eye.
12. The device according to claim 10 , wherein
at least one of:
the processed output is stored within a frame buffer; and
the static vertex buffer is a static texture vertex array.
13. The device according to claim 10 , wherein
the image processing pipeline comprises a graphical processing unit (GPU) comprising at least a vertex shader and a fragment shader, wherein
the received image data is coupled to the fragment shader.
14. The device according to claim 10 , wherein
the static vertex buffer comprises for each vertex in a defined vertex array at least one of:
a texture attribute for each pixel colour within the N2I display together with the vertex's associated coordinates within the N2I display; and
a vertex attribute for each pixel colour within the N2I display established in dependence upon a chief ray angle of a pixel within an acquiring image sensor associated with the vertex in the defined vertex array.
15. The device according to claim 10 , wherein
the static vertex buffer defines which display pixels within the N2I display should be activated for each image pixel within the image to be displayed; wherein
there is a one-to-one mapping of image pixels to display pixels.
16. The device according to claim 15 , wherein
the texture attributes at each vertex within the static vertex buffer define corrections for at least one of optical aberrations and chromatic aberrations within at least one of the optical train and the acquiring image sensor.
17. The device according to claim 10 , wherein
processing the received image data in dependence upon the static vertex buffer also comprises processing the received image data in dependence upon a texture object.
18. The device according to claim 10 , wherein
the vertices comprising the static vertex buffer each define a triangular tesselation of a plane.