Depth-aware reprojection

1. A computing system comprising a display, a sensor subsystem, and a reprojection subsystem configured to:

receive image data to be displayed via the display;

receive a depth map for the image data;

store the image data in a frame buffer;

process the depth map to obtain a blurred depth map;

based upon motion data received from the sensor subsystem, determine a translation to be made to the image data for display; and

for each pixel of a plurality of pixels in the frame buffer, based upon the translation determined,

translate an original ray extending from an original virtual camera location to a corresponding original frame buffer location to a reprojected ray extending from a translated camera location to a reprojected frame buffer location,

determine a location at which the reprojected ray intersects the blurred depth map, and

sample a color of a pixel for display based upon a color in the frame buffer corresponding to the location at which the reprojected ray intersects the blurred depth map.

2. The computing system of claim 1 , wherein the computing system comprises a head-mounted display device.

3. The computing system of claim 1 , wherein the reprojection subsystem is further configured to display via the display the image data after translating the original ray to the reprojected ray.

4. The computing system of claim 1 , wherein the reprojection subsystem is further configured to make a rotation to the image data based upon motion data received from the sensor subsystem.

5. The computing system of claim 1 , wherein the reprojection subsystem is configured to process the depth map to obtain the blurred depth map by omitting background values and weights beyond threshold values during a depth map blur convolution.

6. The computing system of claim 1 , wherein the reprojection subsystem is further configured to determine the location at which the reprojected ray intersects the blurred depth map by

determining an initial depth value on the blurred depth map corresponding to the original frame buffer location,

then determining an initial location on the reprojected ray corresponding to the initial depth value,

then determining an updated depth value on the blurred depth map corresponding to the initial location on the reprojected ray, and

then determining an updated location on the reprojected ray corresponding to the updated depth value.

7. The computing system of claim 6 , wherein the reprojection subsystem is further configured to determine the location at which the reprojected ray intersects the blurred depth map by iteratively repeating the determining of the updated depth value and the determining of the updated location on the reprojected ray.

8. The computing system of claim 6 , wherein the reprojection subsystem is further configured to iteratively repeat the determining of the updated depth value and the determining of the updated location on the reprojected ray a fixed number of times for each pixel in the frame buffer.

9. The computing system of claim 1 , wherein the reprojection subsystem is further configured to reduce a resolution of the depth map.

10. On a computing system comprising a display and a sensor subsystem, a method comprising:

receiving image data to be displayed via the display;

receiving a depth map for the image data;

storing the image data in a frame buffer;

processing the depth map to obtain a blurred depth map;

based upon motion data received from the sensor subsystem, determining a translation to be made to the image data prior to display; and

for each pixel of a plurality of pixels in the frame buffer, based upon the translation determined,

translating an original ray extending from an original virtual camera location to a corresponding original frame buffer location to a reprojected ray extending from a translated camera location to a reprojected frame buffer location,

determining a location at which the reprojected ray intersects the blurred depth map, and

sampling a color of a pixel in the frame buffer for display at the reprojected frame buffer location based upon a color in the frame buffer corresponding to the location at which the reprojected ray intersects the blurred depth map.

11. The method of claim 10 , further comprising displaying via the display the image data after translating the original ray to the reprojected ray for each pixel in the frame buffer.

12. The method of claim 10 , wherein the plurality of pixels comprises each pixel in the frame buffer.

13. The method of claim 10 , wherein determining the location at which the reprojected ray intersects the blurred depth map comprises

determining an initial depth value on the blurred depth map corresponding to the original frame buffer location,

then determining an initial location on the reprojected ray corresponding to the initial depth value,

then determining an updated depth value on the blurred depth map corresponding to the initial location on the reprojected ray, and

then determining an updated location on the reprojected ray corresponding to the updated depth value.

14. The method of claim 13 , wherein determining the location at which the reprojected ray intersects the blurred depth map comprises iteratively repeating the determining of the updated depth value and the determining of the updated location on the reprojected ray.

15. The method of claim 14 , further comprising iteratively repeating the determining of the updated depth value and the determining of the updated location on the reprojected ray a same number of times for each pixel in the frame buffer.

16. The method of claim 10 , further comprising reducing a resolution of the depth map.

17. A head-mounted display device comprising a display, a sensor subsystem, and a reprojection subsystem configured to:

receive image data to be displayed via the display;

receive depth data for the image data;

store the image data in a frame buffer;

process the depth map to obtain a blurred depth map;

based upon motion data received from the sensor subsystem, determine a translation to be made to the image data prior to display; and

for each pixel of a plurality of pixels in the frame buffer, based upon the translation determined,

translate an original ray extending from an original virtual camera location to a corresponding original frame buffer location to a reprojected ray extending from a translated camera location to a reprojected frame buffer location,

determine a location at which the reprojected ray intersects the blurred depth map using successive approximation, and

sample a color of a pixel in the frame buffer for display at the reprojected frame buffer location based upon a color in the frame buffer corresponding to the location at which the reprojected ray intersects the blurred depth map.

18. The head-mounted display device of claim 17 , wherein the reprojection subsystem is configured to determine the location at which the reprojected ray intersects the blurred depth map using successive approximation by

determining an initial depth value on the blurred depth map corresponding to the original frame buffer location,

then determining an initial location on the reprojected ray corresponding to the initial depth value,

then determining an updated depth value on the blurred depth map corresponding to the initial location on the reprojected ray, and

then determining an updated location on the reprojected ray corresponding to the updated depth value.

19. The head-mounted display device of claim 17 , wherein the reprojection subsystem is further configured to iteratively repeat the determining of the updated depth value and the determining of the updated location on the reprojected ray a fixed number of times for each pixel in the frame buffer.

20. The head-mounted display device of claim 17 , wherein the reprojection subsystem is further configured to determine the translation to be made to the image data based on one or more additional frame buffers and corresponding blurred depth maps.