Reducing head mounted display power consumption and heat generation through predictive rendering of content
Systems, methods, and non-transitory computer-readable media are disclosed for selectively rendering augmented reality content based on predictions regarding a user's ability to visually process the augmented reality content. For instance, the disclosed systems can identify eye tracking information for a user at an initial time. Moreover, the disclosed systems can predict a change in an ability of the user to visually process an augmented reality element at a future time based on the eye tracking information. Additionally, the disclosed systems can selectively render the augmented reality element at the future time based on the predicted change in the ability of the user to visually process the augmented reality element.
1. A computer-implemented method comprising:
determining, by an augmented reality system comprising a head mounted display, an eye movement for a user at an initial time;
predicting, from the eye movement at the initial time, a future eye movement of the user in relation to a position of an augmented reality element at a future time utilizing a machine learning model and a future focal point distance of the user from the position of the augmented reality element based on the future eye movement; and
based on a comparison between the future focal point distance of the user and a threshold distance, selectively rendering the augmented reality element for display via the head mounted display or terminating a display of the augmented reality element via the head mounted display.
2. The computer-implemented method of claim 1 , wherein:
determining the eye movement for the user at the initial time comprises determining an initial eye movement vector mapping eye position relative to eye sockets of the user at the initial time; and
predicting the future eye movement of the user comprises utilizing the machine learning model to predict the future eye movement based on the initial eye movement vector.
3. The computer-implemented method of claim 1 , wherein:
predicting the future eye movement comprises predicting that the future focal point distance of the user will be within the threshold distance from the position of the augmented reality element at the future time; and
based on predicting the future focal point distance will be within the threshold distance, rendering the augmented reality element for display via the head mounted display.
4. The computer-implemented method of claim 1 , further comprising:
predicting the future eye movement comprises predicting that the future focal point distance will be outside the threshold distance from the position of the augmented reality element at the future time; and
based on predicting the future focal point distance will be outside the threshold distance, terminating the display of the augmented reality element.
5. The computer-implemented method of claim 1 , wherein predicting the future eye movement of the user comprises predicting one or more of a future change in focal point or a future saccade movement utilizing the machine learning model.
6. The computer-implemented method of claim 1 , wherein predicting the future eye movement of the user comprises utilizing the machine learning model to predict an ability of the user to visually process the augmented reality element at the future time.
7. The computer-implemented method of claim 6 , wherein rendering the augmented reality element for display based on the predicted eye movement comprises rendering the augmented reality element for display based on the predicted ability of the user to visually process the augmented reality element at the future time.
8. An augmented reality system comprising:
a head mounted display;
at least one processor; and
at least one non-transitory computer-readable storage medium storing instructions that, when executed by the at least one processor, cause the augmented reality system to:
determine, by an augmented reality system comprising the head mounted display, an eye movement for a user at an initial time;
predict, from the eye movement at the initial time, a future eye movement of the user in relation to a position of an augmented reality element at a future time utilizing a machine learning model and a future focal point distance of the user from the position of the augmented reality element based on the future eye movement; and
based on a comparison between the future focal point distance of the user and a threshold distance, selectively render the augmented reality element for display via the head mounted display or terminate a display of the augmented reality element via the head mounted display.
9. The augmented reality system of claim 8 , further comprising instructions that, when executed by the at least one processor, cause the augmented reality system to:
determine the eye movement for the user at the initial time by determining an initial eye movement vector mapping eye position relative to eye sockets of the user at the initial time; and
predict the future eye movement of the user by utilizing the machine learning model to predict the future eye movement based on the initial eye movement vector.
10. The augmented reality system of claim 8 , further comprising instructions that, when executed by the at least one processor, cause the augmented reality system to:
predict the future eye movement by predicting that the future focal point distance of the user will be within the threshold distance from the position of the augmented reality element at the future time; and
based on predicting the future focal point distance will be within the threshold distance, render the augmented reality element for display via the head mounted display.
11. The augmented reality system of claim 8 , further comprising instructions that, when executed by the at least one processor, cause the augmented reality system to:
predict the future eye movement comprises predicting that the future focal point distance will be outside the threshold distance from the position of the augmented reality element at the future time; and
based on predicting the future focal point distance will be outside the threshold distance, terminate the display of the augmented reality element.
12. The augmented reality system of claim 8 , further comprising instructions that, when executed by the at least one processor, cause the augmented reality system to predict the future eye movement of the user by predicting one or more of a future change in focal point or a future saccade movement utilizing the machine learning model.
13. The augmented reality system of claim 8 , further comprising instructions that, when executed by the at least one processor, cause the augmented reality system to predict the future eye movement of the user comprises utilizing the machine learning model to predict an ability of the user to visually process the augmented reality element at the future time.
14. The augmented reality system of claim 13 , further comprising instructions that, when executed by the at least one processor, cause the augmented reality system to render the augmented reality element for display based on the predicted eye movement by rendering the augmented reality element for display based on the predicted ability of the user to visually process the augmented reality element at the future time.
15. A non-transitory computer readable medium comprising instructions that, when executed by at least one processor, cause a computer system comprising a head mounted display to:
determine, by an augmented reality system comprising the head mounted display, an eye movement for a user at an initial time;
predict, from the eye movement at the initial time, a future eye movement of the user in relation to a position of an augmented reality element at a future time utilizing a machine learning model and a future focal point distance of the user from the position of the augmented reality element based on the future eye movement; and
based on a comparison between the future focal point distance of the user and a threshold distance, selectively render the augmented reality element for display via the head mounted display or terminate a display of the augmented reality element via the head mounted display.
16. The non-transitory computer readable medium of claim 15 , further comprising instructions that, when executed by the at least one processor, cause the computer system to:
determine the eye movement for the user at the initial time by determining an initial eye movement vector mapping eye position relative to eye sockets of the user at the initial time; and
predict the future eye movement of the user by utilizing the machine learning model to predict the future eye movement based on the initial eye movement vector.
17. The non-transitory computer readable medium of claim 15 , further comprising instructions that, when executed by the at least one processor, cause the computer system to:
predict the future eye movement by predicting that the future focal point distance of the user will be within the threshold distance of from the position of the augmented reality element at the future time; and
based on predicting the future focal point distance will be within the threshold distance, render the augmented reality element for display via the head mounted display.
18. The non-transitory computer readable medium of claim 15 , further comprising instructions that, when executed by the at least one processor, cause the computer system to:
predict the future eye movement comprises predicting that the future focal point distance will be outside the threshold distance from the position of the augmented reality element at the future time; and
based on predicting the future focal point distance will be outside the threshold distance, terminate the display of the augmented reality element.
19. The non-transitory computer readable medium of claim 15 , further comprising instructions that, when executed by the at least one processor, cause the computer system to predict the future eye movement of the user by predicting one or more of a future change in focal point or a future saccade movement utilizing the machine learning model.
20. The non-transitory computer readable medium of claim 15 , further comprising instructions that, when executed by the at least one processor, cause the computer system to predict the future eye movement of the user comprises utilizing the machine learning model to predict an ability of the user to visually process the augmented reality element at the future time.