Foveated domain storage and processing

1. An electronic device for foveated domain storage and processing, the electronic device comprising:

a memory; and

a processor coupled to the memory, the processor configured to:

perform head tracking and eye tracking;

generate a foveated image from an original image based on the head tracking and the eye tracking, wherein the processor is configured to use a selected one of multiple warping functions based on whether there is a change in a head position, a change in a gaze direction, or changes in both the head position and the gaze direction; and

store the foveated image using one of: a tile-based method or a frame-based method,

wherein, in the tile-based method, the processor is further configured to:

divide the original image into a plurality of tiles; and

scale each of the plurality of tiles based on a distance of each tile from the gaze direction, wherein the processor is configured to generate the foveated image based on the plurality of scaled tiles, and

wherein, in the frame-based method, the processor is further configured to apply a scale factor for each pixel based on a horizontal distance and a vertical distance from the gaze direction.

2. The electronic device of claim 1 , wherein the processor is further configured to convert the original image to the foveated image using a foveation function for each pixel.

3. The electronic device of claim 1 , wherein the selected warping function computes a warp in a foveated domain without intermediate conversion to a full format.

4. The electronic device of claim 1 , wherein the selected warping function uses a different address calculation for an input image and an output image to accommodate for the change in the gaze direction.

5. The electronic device of claim 1 , wherein the selected warping function uses an affine matrix when the head position changes.

6. The electronic device of claim 1 , wherein the selected warping function is defined by either a look-up table (LUT)-based mesh or a radial distortion function if a color aberration or lens distortion correction is required while an output image is streamed to display in full resolution and an input image is in a foveated format.

7. The electronic device of claim 6 , wherein the processor is further configured to store an intermediate image in a foveated format if compensation for the change in the head position, the color aberration, or the lens distortion is required while both the input image and the output image are stored in full format.

8. A method for foveated storage and processing comprising:

performing head tracking and eye tracking;

generating a foveated image from an original image based on the head tracking and the eye tracking, wherein a selected one of multiple warping functions is used based on whether there is a change in a head position, a change in a gaze direction, or changes in both the head position and the gaze direction; and

storing the foveated image using one of: a tile-based method or a frame-based method,

wherein the tile-based method comprises:

dividing the original image into a plurality of tiles; and

scaling each of the plurality of tiles based on a distance of each tile from the gaze direction, wherein the foveated image is further generated based on the plurality of scaled tiles, and

wherein the frame-based method comprises applying a scale factor for each pixel based on a horizontal distance and a vertical distance from the gaze direction.

9. The method of claim 8 , further comprising:

converting the original image to the foveated image using a foveation function for each pixel.

10. The method of claim 8 , wherein the selected warping function computes a warp in a foveated domain without intermediate conversion to a full format.

11. The method of claim 8 , wherein the selected warping function uses a different address calculation for an input image and an output image to accommodate for the change in the gaze direction.

12. The method of claim 8 , wherein the selected warping function uses an affine matrix when the head position changes.

13. The method of claim 8 , wherein the selected warping function is defined by either a look-up table (LUT)-based mesh or a radial distortion function if a color aberration or lens distortion correction is required while an output image is streamed to display in full resolution and an input image is in a foveated format.

14. The method of claim 13 , further comprising:

storing an intermediate image in a foveated format if compensation for the change in the head position, the color aberration, or the lens distortion is required while both the input image and the output image are stored in full format.

15. A non-transitory computer readable medium embodying a computer program, the computer program comprising computer readable program code that when executed causes at least one processor to:

perform head tracking and eye tracking;

generate a foveated image from an original image based on the head tracking and the eye tracking, wherein a selected one of multiple warping functions is used based on whether there is a change in a head position, a change in a gaze direction, or changes in both the head position and the gaze direction; and

store the foveated image using one of: a tile-based method or a frame-based method,

wherein, in the tile-based method, the computer readable program code causes the at least one processor to:

divide the original image into a plurality of tiles; and

scale each of the plurality of tiles based on a distance of each tile from the gaze direction, wherein the foveated image is further generated based on the plurality of scaled tiles, and

wherein, in the frame-based method, the computer readable program code causes the at least one processor to apply a scale factor for each pixel based on a horizontal distance and a vertical distance from the gaze direction.

16. The non-transitory computer readable medium of claim 15 , wherein the computer program further comprises computer readable program code that when executed causes at least one processor to:

convert the original image to the foveated image using a foveation function for each pixel.

17. The non-transitory computer readable medium of claim 15 , wherein the selected warping function computes a warp in a foveated domain without intermediate conversion to a full format.

18. The non-transitory computer readable medium of claim 15 , wherein the selected warping function uses a different address calculation for an input image and an output image to accommodate for the change in the gaze direction.

19. The non-transitory computer readable medium of claim 15 , wherein the selected warping function uses an affine matrix when the head position changes.

20. The non-transitory computer readable medium of claim 15 , wherein the selected warping function is defined by either a look-up table (LUT)-based mesh or a radial distortion function if a color aberration or lens distortion correction is required while an output image is streamed to display in full resolution and an input image is in a foveated format.

21. The non-transitory computer readable medium of claim 20 , wherein the computer program further comprises computer readable program code that when executed causes at least one processor to:

store an intermediate image in a foveated format if compensation for the change in the head position, the color aberration, or the lens distortion is required while both the input image and the output image are stored in full format.