Late update of eye tracking information to GPU for fast foveated rendering

1. A method, comprising:

generating primitives of a scene for a video frame;

receiving gaze tracking information for at least one eye of a user in a frame period;

predicting a landing point on a display corresponding to a gaze direction of the at least one eye in the frame period based on the gaze tracking information; and

rendering the video frame in the frame period based on the primitives and the landing point that is predicted,

wherein the receiving the gaze tracking information includes tracking a first eye and a second eye of the user to increase a number of sample points used to determine the gaze tracking information.

2. The method of claim 1 , wherein the rendering the video frame includes:

performing one or more shader operations to generate pixel data for a plurality of pixels of the display based on the primitives and the landing point that is predicted.

3. The method of claim 2 , further comprising:

storing the pixel data in a frame buffer; and

scanning out in a subsequent frame period the pixel data from the frame buffer to the display.

4. The method of claim 3 ,

wherein the subsequent frame period is a next frame period.

5. The method of claim 1 , wherein the rendering the video frame includes:

rendering a foveal region of high resolution in the video frame, wherein the foveal region is centered about the landing point that is predicted.

6. The method of claim 1 ,

detecting that the at least one eye of the user is experiencing a saccade; and

performing the predicting the landing point based on the saccade.

7. The method of claim 6 , further comprising:

tracking movement of the at least one eye of the user;

determining velocity of the movement based on the tracking;

determining that the at least one eye is in the saccade upon the velocity reaching a threshold velocity; and

predicting the landing point corresponding to a direction of the at least one eye for the saccade.

8. The method of claim 1 , further comprising:

executing in a previous frame period occurring before the frame period an application to generate the primitives of the scene for the video frame.

9. The method of claim 1 ,

wherein the display is disposed within a head mounted display.

10. The method of claim 1 ,

wherein the landing point is predicted with a decrease in error and an increase in accuracy when tracking the first eye and second eye to increase the number of sample points.

11. The method of claim 1 ,

wherein the first eye and the second eye are separately and simultaneously tracked.

12. A non-transitory computer-readable medium storing a computer program for performing a method, the computer-readable medium comprising:

program instructions for generating primitives of a scene for a video frame;

program instructions for receiving gaze tracking information for at least one eye of a user in a frame period;

program instructions for predicting a landing point on a display corresponding to a gaze direction of the at least one eye in the frame period based on the gaze tracking information; and

program instructions for rendering the video frame in the frame period based on the primitives and the landing point that is predicted,

wherein the program instructions for receiving the gaze tracking information includes program instructions for tracking a first eye and a second eye of the user to increase a number of sample points used to determine the gaze tracking information.

13. The non-transitory computer-readable medium of claim 12 , wherein the program instructions for rendering the video frame includes:

program instructions for performing one or more shader operations to generate pixel data for a plurality of pixels of the display based on the primitives and the landing point that is predicted.

14. The non-transitory computer-readable medium of claim 12 , wherein the program instructions for rendering the video frame includes:

program instructions for rendering a foveal region of high resolution in the video frame, wherein the foveal region is centered about the landing point that is predicted.

15. The non-transitory computer-readable medium of claim 12 , further comprising:

program instructions for detecting that the at least one eye of the user is experiencing a saccade; and

program instructions for performing the predicting the landing point based on the saccade.

16. The non-transitory computer-readable medium of claim 15 , further comprising:

program instructions for tracking movement of the at least one eye of the user;

program instructions for determining velocity of the movement based on the tracking;

program instructions for determining that the at least one eye is in the saccade upon the velocity reaching a threshold velocity; and

program instructions for predicting the landing point corresponding to a direction of the at least one eye for the saccade.

17. The non-transitory computer-readable medium of claim 12 , further comprising:

program instructions for executing in a previous frame period occurring before the frame period an application to generate the primitives of the scene for the video frame.

18. The non-transitory computer-readable medium of claim 12 ,

wherein the landing point is predicted with a decrease in error and an increase in accuracy when tracking the first eye and second eye to increase the number of sample points.

19. The non-transitory computer-readable medium of claim 12 ,

wherein in the method the first eye and the second eye are separately and simultaneously tracked.

20. A computer system comprising:

a processor;

memory coupled to the processor and having stored therein instructions that, if executed by the computer system, cause the computer system to execute a method, comprising:

generating primitives of a scene for a video frame;

receiving gaze tracking information for at least one eye of a user in a frame period;

predicting a landing point on a display corresponding to a gaze direction of the at least one eye in the frame period based on the gaze tracking information; and

rendering the video frame in the frame period based on the primitives and the landing point that is predicted,

wherein the receiving the gaze tracking information includes tracking a first eye and a second eye of the user to increase a number of sample points used to determine the gaze tracking information.

21. The computer system of claim 20 , wherein in the method the rendering the video frame includes:

performing one or more shader operations to generate pixel data for a plurality of pixels of the display based on the primitives and the landing point that is predicted.

22. The computer system of claim 20 , wherein in the method the rendering the video frame includes:

rendering a foveal region of high resolution in the video frame, wherein the foveal region is centered about the landing point that is predicted.

23. The computer system of claim 20 , the method further including:

detecting that the at least one eye of the user is experiencing a saccade; and

performing the predicting the landing point based on the saccade.

24. The computer system of claim 20 , the method further including:

executing in a previous frame period occurring before the frame period an application to generate the primitives of the scene for the video frame.

25. The computer system of claim 20 ,

wherein in the method the landing point is predicted with a decrease in error and an increase in accuracy when tracking the first eye and second eye to increase the number of sample points.

26. The computer system of claim 20 ,

wherein in the method the first eye and the second eye are separately and simultaneously tracked.