Method for simulating natural perception in virtual and augmented reality scenes

1. A computer-implemented method comprising:

detecting, for a computing device, an initial visual locus point directed away from a user in an initial direction with respect to a light source fora three-dimensional virtual or augmented reality scene;

based on detecting the initial visual locus point, generating an initial view of the three-dimensional virtual or augmented reality scene utilizing a set of pixel values for a set of images of the three-dimensional virtual or augmented reality scene that are mapped to a set of three-dimensional vectors;

detecting, for the computing device, a subsequent visual locus point directed away from the user in a subsequent direction with respect to the light source for the three-dimensional virtual or augmented reality scene based on inputs from at least one of a gyroscope, a magnetometer, or an accelerometer of the computing device; and

based on detecting the subsequent visual locus point:

modifying a brightness of the set of pixel values for the set of images of the three-dimensional virtual or augmented reality scene that are mapped to the set of three-dimensional vectors, by applying a fragment shader to brighten or darken one or more pixels of the set of pixel values; and

generating a subsequent view of the three-dimensional virtual or augmented reality scene according to the modified brightness of the set of pixel values.

2. The computer-implemented method of claim 1 , wherein the light source comprises a real light source from a real environment or a synthetic light source from a synthetic environment.

3. The computer-implemented method of claim 1 , wherein generating the subsequent view of the three-dimensional virtual or augmented reality scene according to the modified brightness of the set of pixel values comprises modifying a brightness of the three-dimensional virtual or augmented reality scene.

4. The computer-implemented method of claim 1 , wherein detecting the subsequent visual locus point comprises determining a point that contains a vector through a center of a screen to the three-dimensional virtual or augmented reality scene as oriented by the inputs from the at least one of the gyroscope, the magnetometer, or the accelerometer.

5. The computer-implemented method of claim 1 , wherein:

generating the initial view of the three-dimensional virtual or augmented reality scene comprises determining a first set of pixel values associated with a particular brightness and corresponding to the set of pixel values to simulate a shorter exposure-frustrum orientation for a camera; and

generating the subsequent view of the three-dimensional virtual or augmented reality scene comprises determining a second set of pixel values associated with a different particular brightness and corresponding to the set of pixel values to simulate a longer exposure-frustrum orientation for the camera.

6. The computer-implemented method of claim 1 , wherein:

detecting the initial visual locus point corresponding to the initial direction with respect to the light source for the three-dimensional virtual or augmented reality scene comprises detecting the initial visual locus point is directed toward the light source; and

detecting the subsequent visual locus point in the subsequent direction with respect to the light source for the three-dimensional virtual or augmented reality scene comprises detecting the subsequent visual locus point is directed away from the light source.

7. The computer-implemented method of claim 1 , further comprising:

selecting, from the set of images of the three-dimensional virtual or augmented reality scene that are mapped to the set of three-dimensional vectors, a representative image that approximates a set of photometric parameters at the subsequent visual locus point; and

wherein modifying the brightness of the set of pixel values for the set of images of the three-dimensional virtual or augmented reality scene comprises modifying the brightness of the set of pixel values based on the set of photometric parameters approximated by the representative image.

8. The computer-implemented method of claim 1 , wherein generating the initial view of the three-dimensional virtual or augmented reality scene utilizing the set of pixel values comprises compo siting pixel values from a plurality of real images from the set of images of the three-dimensional virtual or augmented reality scene that are mapped to the set of three-dimensional vectors into a composite image for the three-dimensional virtual or augmented reality scene.

9. The computer-implemented method of claim 1 , further comprising generating the initial view of the three-dimensional virtual or augmented reality scene by:

calculating a graphical projection of the three-dimensional virtual or augmented reality scene based on a plurality of images from the set of images of the three-dimensional virtual or augmented reality scene that are mapped to the set of three-dimensional vectors; and

determining a discretized ray direction of the three-dimensional virtual or augmented reality scene for each image of the plurality of images through the initial visual locus point.

10. A non-transitory computer readable storage medium comprising instructions that, when executed by at least one processor, cause a computing device to:

detect, for a computing device, an initial visual locus point directed away from a user in an initial direction with respect to a light source for a three-dimensional virtual or augmented reality scene;

based on detecting the initial visual locus point, generate an initial view of the three-dimensional virtual or augmented reality scene utilizing a set of pixel values for a set of images of the three-dimensional virtual or augmented reality scene that are mapped to a set of three-dimensional vectors;

detect, for the computing device, a subsequent visual locus point directed away from the user in a subsequent direction with respect to the light source for the three-dimensional virtual or augmented reality scene based on inputs from at least one of a gyroscope, a magnetometer, or an accelerometer of the computing device; and

based on detecting the subsequent visual locus point:

modify a brightness of the set of pixel values for the set of images of the three-dimensional virtual or augmented reality scene that are mapped to the set of three-dimensional vectors, by applying a fragment shader to the set of pixel values to brighten or darken one or more pixels corresponding to the set of pixel values; and

generate a subsequent view of the three-dimensional virtual or augmented reality scene according to the modified brightness of the set of pixel values.

11. The non-transitory computer readable storage medium of claim 10 , wherein the light source comprises a real light source from a real environment or a synthetic light source from a synthetic environment.

12. The non-transitory computer readable storage medium of claim 10 , further comprising instructions that, when executed by the at least one processor, cause the computing device to modify the brightness of the set of pixel values by simulating a longer exposure of a camera or a shorter exposure of the camera.

13. The non-transitory computer readable storage medium of claim 10 , further comprising instructions that, when executed by the at least one processor, cause the computing device to detect the subsequent visual locus point by determining a point that contains a vector through a center of a screen to the three-dimensional virtual or augmented reality scene as oriented by the inputs from the at least one of the gyroscope, the magnetometer, or the accelerometer.

14. The non-transitory computer readable storage medium of claim 10 , further comprising instructions that, when executed by the at least one processor, cause the computing device to:

generate the initial view of the three-dimensional virtual or augmented reality scene by determining a first set of pixel values associated with a particular brightness and corresponding to the set of pixel values to simulate a shorter exposure-frustrum orientation for a camera; and

generate the subsequent view of the three-dimensional virtual or augmented reality scene by determining a second set of pixel values associated with a different particular brightness and corresponding to the set of pixel values to simulate a longer exposure-frustrum orientation for the camera.

15. The non-transitory computer readable storage medium of claim 10 , further comprising instructions that, when executed by the at least one processor, cause the computing device to:

detect the initial visual locus point corresponding to the initial direction with respect to the light source for the three-dimensional virtual or augmented reality scene comprises detecting the initial visual locus point is directed toward the light source; and

detect the subsequent visual locus point in the subsequent direction with respect to the light source for the three-dimensional virtual or augmented reality scene comprises detecting the subsequent visual locus point is directed away from the light source.

16. A computing device comprising:

at least one processor; and

at least one non-transitory computer readable medium storing instructions thereon that, when executed by the at least one processor, cause the computing device to:

detect, for the computing device, an initial visual locus point directed away from a user in an initial direction with respect to a light source for a three-dimensional virtual or augmented reality scene;

based on detecting the initial visual locus point, generate an initial view of the three-dimensional virtual or augmented reality scene utilizing a set of pixel values for a set of images of the three-dimensional virtual or augmented reality scene that are mapped to a set of three-dimensional vectors;

detect, for the computing device, a subsequent visual locus point directed away from the user in a subsequent direction with respect to the light source for the three-dimensional virtual or augmented reality scene based on inputs from at least one of a gyroscope, a magnetometer, or an accelerometer of the computing device; and

based on detecting the subsequent visual locus point:

modify a brightness of the set of pixel values for the set of images of the three-dimensional virtual or augmented reality scene that are mapped to the set of three-dimensional vectors, by applying a fragment shader to brighten or darken one or more pixels of the set of pixel values; and

generate a subsequent view of the three-dimensional virtual or augmented reality scene according to the modified brightness of the set of pixel values.

17. The computing device of claim 16 , wherein generating the subsequent view of the three-dimensional virtual or augmented reality scene according to the modified brightness of the set of pixel values comprises modifying a brightness of the three-dimensional virtual or augmented reality scene.

18. The computing device of claim 16 , further comprising instructions that, when executed by the at least one processor, cause the computing device to:

select, from the set of images of the three-dimensional virtual or augmented reality scene that are mapped to the set of three-dimensional vectors, a representative image that approximates a set of photometric parameters at the subsequent visual locus point; and

wherein modifying the brightness of the set of pixel values for the set of images of the three-dimensional virtual or augmented reality scene comprises modifying the brightness of the set of pixel values based on the set of photometric parameters approximated by the representative image.

19. The computing device of claim 16 , further comprising instructions that, when executed by the at least one processor, cause the computing device to generate the initial view of the three-dimensional virtual or augmented reality scene utilizing the set of pixel values by compositing pixel values from a plurality of real images from the set of images of the three-dimensional virtual or augmented reality scene that are mapped to the set of three-dimensional vectors into a composite image for the three-dimensional virtual or augmented reality scene.

20. The computing device of claim 16 , further comprising instructions that, when executed by the at least one processor, cause the computing device to render the initial view of the three-dimensional virtual or augmented reality scene by:

calculate a graphical projection of the three-dimensional virtual or augmented reality scene based on a plurality of images from the set of images of the three-dimensional virtual or augmented reality scene that are mapped to the set of three-dimensional vectors; and

determine a discretized ray direction of the three-dimensional virtual or augmented reality scene for each image of the plurality of images through the initial visual locus point.