DEPTH BASED FOVEATED RENDERING FOR DISPLAY SYSTEMS
Methods and systems for depth-based foveated rendering in the display system are disclosed. The display system may be an augmented reality display system configured to provide virtual content on a plurality of depth planes using different wavefront divergence. Some embodiments include determining a fixation point of a user's eyes. Location information associated with a first virtual object to be presented to the user via a display device is obtained. A resolution-modifying parameter of the first virtual object is obtained. A particular resolution at which to render the first virtual object is identified based on the location information and the resolution-modifying parameter of the first virtual object. The particular resolution is based on a resolution distribution specifying resolutions for corresponding distances from the fixation point. The first virtual object rendered at the identified resolution is presented to the user via the display system.
1 . A wearable display system comprising:
a display configured to output light at a plurality of depths to present virtual content to a user of the wearable display system;
one or more processors communicatively coupled to the display; and
one or more computer storage media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
determining a three-dimensional (3D) fixation point of eyes of the user;
determining a 3D virtual object position at which a virtual object is to be presented within a field of view (FOV) of the user via the display;
determining an angular distance of the virtual object position from the fixation point and a depth difference of the virtual object position from the fixation point;
determining a particular resolution at which to render the virtual object based at least partly on position of the angular distance and the depth difference on a resolution distribution that designates resolution as a function of angular distance and depth difference such that the resolution varies with depth difference for a particular angular distance, wherein the resolution distribution includes: a) a foveal zone of high resolution proximal to the fixation point, b) a low resolution zone surrounding the foveal zone, and c) a rolloff providing a continuous drop-off in resolution between the foveal zone and the low resolution zone; and
causing the display to present the virtual object at the virtual object position and at the particular resolution.
2 . The wearable display system of claim 1 , wherein the resolution distribution is a multivariate normal distribution.
3 . The wearable display system of claim 1 , wherein the resolution distribution provides that the particular resolution decreases with increase in the depth difference.
4 . The wearable display system of claim 1 , wherein the rolloff is a Gaussian rolloff.
5 . The wearable display system of claim 1 , wherein resolution is substantially constant for angular distances within the foveal zone for a particular depth difference.
6 . The wearable display system of claim 1 , wherein the operations further comprise:
determining a type of the virtual object; and
based on the type of the virtual object, modifying the resolution distribution to adjust at least one of a shape of the rolloff or an angular distance encompassed by the foveal zone.
7 . The wearable display system of claim 1 , wherein input from the user is employed to modify the resolution distribution to adjust at least one of a shape of the rolloff or an angular distance encompassed by the foveal zone.
8 . The wearable display system of claim 1 , wherein detected eye movement of the user is employed to modify the resolution distribution to adjust at least one of a shape of the rolloff or an angular distance encompassed by the foveal zone.
9 . The wearable display system of claim 8 , wherein a detected speed of a saccade of the eye of the user is used to adjust the angular distance encompassed by the foveal zone.
10 . Computer-readable media storing instructions which, when executed by at least one processor, instruct the at least one processor to perform operations for presenting virtual content through a wearable display system, the operations comprising:
determining a three-dimensional (3D) fixation point of eyes of a user of the wearable display system comprising a display configured to output light at a plurality of depths to present the virtual content to the user;
determining a 3D virtual object position at which a virtual object is to be presented within a field of view (FOV) of the user via the display;
determining an angular distance of the virtual object position from the fixation point and a depth difference of the virtual object position from the fixation point;
determining a particular resolution at which to render the virtual object based at least partly on position of the angular distance and the depth difference on a resolution distribution that designates resolution as a function of angular distance and depth difference such that the resolution varies with depth difference for a particular angular distance, wherein the resolution distribution includes: a) a foveal zone of high resolution proximal to the fixation point, b) a low resolution zone surrounding the foveal zone, and c) a rolloff providing a continuous drop-off in resolution between the foveal zone and the low resolution zone; and
causing the display to present the virtual object at the virtual object position and at the particular resolution.
11 . A computer-implemented method for presenting virtual content through a wearable display system, the method comprising:
determining a three-dimensional (3D) fixation point of eyes of a user of the wearable display system comprising a display configured to output light at a plurality of depths to present the virtual content to the user;
determining a 3D virtual object position at which a virtual object is to be presented within a field of view (FOV) of the user via the display;
determining an angular distance of the virtual object position from the fixation point and a depth difference of the virtual object position from the fixation point;
determining a particular resolution at which to render the virtual object based at least partly on position of the angular distance and the depth difference on a resolution distribution that designates resolution as a function of angular distance and depth difference such that the resolution varies with depth difference for a particular angular distance, wherein the resolution distribution includes: a) a foveal zone of high resolution proximal to the fixation point, b) a low resolution zone surrounding the foveal zone, and c) a rolloff providing a continuous drop-off in resolution between the foveal zone and the low resolution zone; and
causing the display to present the virtual object at the virtual object position and at the particular resolution.
12 . The computer-implemented method of claim 11 , wherein the resolution distribution is a multivariate normal distribution.
13 . The computer-implemented method of claim 11 , wherein the resolution distribution provides that the particular resolution decreases with increase in the depth difference.
14 . The computer-implemented method of claim 11 , wherein the rolloff is a Gaussian rolloff.
15 . The computer-implemented method of claim 11 , wherein resolution is substantially constant for angular distances within the foveal zone for a particular depth difference.
16 . The computer-implemented method of claim 11 , wherein the operations further comprise:
determining a type of the virtual object; and
based on the type of the virtual object, modifying the resolution distribution to adjust at least one of a shape of the rolloff or an angular distance encompassed by the foveal zone.
17 . The computer-implemented method of claim 11 , wherein input from the user is employed to modify the resolution distribution to adjust at least one of a shape of the rolloff or an angular distance encompassed by the foveal zone.
18 . The computer-implemented method of claim 11 , wherein detected eye movement of the user is employed to modify the resolution distribution to adjust at least one of a shape of the rolloff or an angular distance encompassed by the foveal zone.
19 . The computer-implemented method of claim 18 , wherein a detected speed of a saccade of the eye of the user is used to adjust the angular distance encompassed by the foveal zone.