Distributed foveated rendering based on user gaze

1. A method for distributed foveated rendering between a remote computer and an end user device that presents images on a display based on user gaze, the method implemented on the end user device and comprising:

receiving, from the remote computer, a background image and a foreground image, wherein:

the background image has a lower resolution than the foreground image and has first dimensions that are smaller than an image display region of the display, and

the foreground image has second dimensions that are equal to a foveated region located within the image display region;

generating a composite image by scaling up the background image to the image display region and by overlaying the foreground image based on gaze data of a user of the end user device;

warping the composite image; and

presenting the warped composite image in the image display region of the display; and

wherein the method further comprises

determining a brightness difference between a first pixel from the overlaid foreground image and a second pixel from the scaled up background image, wherein the first pixel and the second pixel correspond to a same object visible in the composite image; and

adjusting a brightness of the first pixel based on a comparison of the brightness difference to a predefined threshold.

2. The method of claim 1 , wherein generating the composite image further comprises:

blending an internal region of the scaled up background image and a boundary region of the overlaid foreground image, wherein the internal region and the boundary region are overlapping.

3. The method of claim 1 , wherein the object is an edge, and further comprising:

selecting the first pixel from pixels of the overlaid foreground image for comparison to the second pixel based on a determination that the first pixel belongs to the edge, and wherein adjusting the brightness comprises reducing the brightness of the first pixel.

4. The method of claim 1 , wherein the end user device comprises a virtual reality headset, and wherein the composite image is warped based on a warp compensation for a lens of the virtual reality headset.

5. The method of claim 4 , wherein warping the composite image comprises repositioning a presentation boundary of the scaled up background image based on orientation data of the virtual reality headset.

6. The method of claim 1 , further comprising:

sending, to the remote computer, the gaze data of the user and orientation data of the end user device, wherein:

the background image is rendered by the remote computer based on the orientation data,

the foreground image is rendered by the remote computer based on the gaze data, and

warping the composite image comprises repositioning a presentation boundary of the scaled up background image based on updated orientation data of the end user device.

7. A non-transitory computer-readable storage medium storing instructions that, upon execution on an end user device, configure the end user device to perform operations comprising:

receiving, from a remote computer, a background image and a foreground image, wherein:

the background image has a lower resolution than the foreground image and has first dimensions that are smaller than an image display region of a display, and

the foreground image has second dimensions that are equal to a foveated region located within the image display region;

generating a composite image by scaling up the background image to the image display region and by overlaying the foreground image based on gaze data of a user of the end user device;

warping the composite image; and

presenting the warped composite image in the image display region of the display;

wherein the operations further comprise:

determining a brightness difference between a first pixel from the overlaid foreground image and a second pixel from the scaled up background image, wherein the first pixel and the second pixel correspond to a same object visible in the composite image; and

adjusting a brightness of the first pixel based on a comparison of the brightness difference to a predefined threshold.

8. The non-transitory computer-readable storage medium of claim 7 , wherein generating the composite image further comprises:

blending an internal region of the scaled up background image and a boundary region of the overlaid foreground image, wherein the internal region and the boundary region are overlapping.

9. The non-transitory computer-readable storage medium of claim 7 , wherein the object is an edge, and wherein the operations further comprise:

selecting the first pixel from pixels of the overlaid foreground image for comparison to the second pixel based on a determination that the first pixel belongs to the edge, and wherein adjusting the brightness comprises reducing the brightness of the first pixel.

10. The non-transitory computer-readable storage medium of claim 7 , wherein the end user device comprises a virtual reality headset, and wherein the composite image is warped based on a warp compensation for a lens of the virtual reality headset.

11. The non-transitory computer-readable storage medium of claim 10 , wherein warping the composite image comprises repositioning a presentation boundary of the scaled up background image based on orientation data of the virtual reality headset.

12. The non-transitory computer-readable storage medium of claim 7 , wherein the operations further comprise:

sending, to the remote computer, the gaze data of the user and orientation data of the end user device, wherein:

the background image is rendered by the remote computer based on the orientation data,

the foreground image is rendered by the remote computer based on the gaze data, and

warping the composite image comprises repositioning a presentation boundary of the scaled up background image based on updated orientation data of the end user device.

13. A computer system comprising:

a remote computer; and

an end user device configured to present image on a display based on user gaze, wherein the end user device comprises memory storing instructions that, upon execution on the end user device, configure the end user device to perform operations comprising:

receiving, from the remote computer, a background image and a foreground image, wherein:

the background image has a lower resolution than the foreground image and has first dimensions that are smaller than an image display region of the display, and

the foreground image has second dimensions that are equal to a foveated region located within the image display region;

generating a composite image by scaling up the background image to the image display region and by overlaying the foreground image based on gaze data of a user of the end user device;

warping the composite image; and

presenting the warped composite image in the image display region of the display;

wherein the operations further comprise:

determining a brightness difference between a first pixel from the overlaid foreground image and a second pixel from the scaled up background image, wherein the first pixel and the second pixel correspond to a same object visible in the composite image; and

adjusting a brightness of the first pixel based on a comparison of the brightness difference to a predefined threshold.

14. The computer system of claim 13 , wherein the end user device comprises a virtual reality headset, and wherein the composite image is warped based on a warp compensation for a lens of the virtual reality headset.

15. The computer system of claim 14 , wherein warping the composite image comprises repositioning a presentation boundary of the scaled up background image based on orientation data of the virtual reality headset.

16. The computer system of claim 13 , wherein the operations of the end user device further comprise sending, to the remote computer, the gaze data of the user and orientation data of the end user device, wherein remote computer comprises second memory storing second instructions that, upon execution on the remote computer, configure the remote computer to perform second operations comprising:

rendering the background image based on the orientation data;

rendering the foreground image based on the gaze data; and

sending the background image and the foreground image to the end user device.

17. The computer system of claim 16 , wherein warping the composite image comprises repositioning a presentation boundary of the scaled up background image based on updated orientation data of the end user device.

18. The computer system of claim 13 , wherein generating the composite image further comprises:

blending an internal region of the scaled up background image and a boundary region of the overlaid foreground image, wherein the internal region and the boundary region are overlapping.