Reducing head mounted display power consumption and heat generation through predictive rendering of content
Systems, methods, and non-transitory computer-readable media are disclosed for selectively rendering augmented reality content based on predictions regarding a user's ability to visually process the augmented reality content. For instance, the disclosed systems can identify eye tracking information for a user at an initial time. Moreover, the disclosed systems can predict a change in an ability of the user to visually process an augmented reality element at a future time based on the eye tracking information. Additionally, the disclosed systems can selectively render the augmented reality element at the future time based on the predicted change in the ability of the user to visually process the augmented reality element.
1. A computer-implemented method comprising:
predicting, from a movement of a user of a head mounted display and utilizing a machine learning model, a future viewpoint of the user;
based on predicting that the user-future viewpoint will include a location associated with a virtual graphical object, rendering the virtual graphical object for display via the head mounted display;
predicting, from another movement of the user of the head mounted display and utilizing the machine learning model, an additional future viewpoint of the user; and
based on predicting that the additional future viewpoint will not include an additional location associated with an additional virtual graphical object, avoiding rendering of the additional virtual graphical object via the head mounted display.
2. The computer-implemented method of claim 1 , further comprising, after rendering the virtual graphical object for display via the head mounted display:
predicting, from a second movement of the user of the head mounted display and utilizing the machine learning model, a second future viewpoint of the user; and
based on predicting that the second future viewpoint will not include the location associated with virtual graphical object, terminate rendering of the virtual graphical object via the head mounted display.
3. The computer-implemented method of claim 1 , further comprising, after avoiding rendering the additional virtual graphical object for display via the head mounted display:
predicting, from a third movement of the user of the head mounted display and utilizing the machine learning model, a third future viewpoint of the user; and
based on predicting that the third future viewpoint will include the location associated with the additional virtual graphical object, rendering the additional virtual object for display via the head mounted display.
4. The computer-implemented method of claim 1 , wherein the movement of the user comprises at least one of a body movement of the user and a head movement of the user.
5. The computer-implemented method of claim 1 , further comprising determining the movement of the user based on sensor data received from at least one sensor of the head mounted display.
6. The computer-implemented method of claim 1 , further comprising:
predicting, from a third movement of the user of the head mounted display and utilizing the machine learning model, a third future viewpoint of the user; and
based on the third future viewpoint of the user, modifying a frame rate of one or more virtual graphical objects rendered for display via the head mounted display.
7. The computer-implemented method of claim 1 , further comprising:
predicting, from a fourth movement of the user of the head mounted display and utilizing the machine learning model, a fourth future viewpoint of the user; and
based on the fourth future viewpoint of the user, modifying a resolution of one or more virtual graphical objects rendered for display via the head mounted display.
8. The computer-implemented method of claim 1 , further comprising:
predicting, from a fifth movement of the user of the head mounted display and utilizing the machine learning model, a fifth future viewpoint of the user; and
based on the fifth future viewpoint of the user, enabling or disabling one or more sensors of the head mounted display.
9. The computer-implemented method of claim 1 , further comprising:
predicting, from a sixth movement of the user of the head mounted display and utilizing the machine learning model, a sixth future viewpoint of the user; and
based on the sixth future viewpoint of the user, enabling or disabling one or more graphics pipeline processes.
10. The computer-implemented method of claim 1 , further comprising predicting a future position of the user based on the movement of the user, and wherein the predicting the future viewpoint of the user is further based on the future position.
11. An artificial reality system comprising:
a head mounted display;
at least one processor; and
at least one non-transitory computer-readable storage medium storing instructions that, when executed by the at least one processor, cause the artificial reality system to:
predict, from a movement of a user of the head mounted display and utilizing a machine learning model, a future viewpoint of the user;
based on predicting that the future viewpoint will include a location associated with a virtual graphical object, render the virtual graphical object for display via the head mounted display;
predict, from another movement of the user of the head mounted display and utilizing the machine learning model, an additional future viewpoint of the user; and
based on predicting that the additional future viewpoint will not include an additional location associated with an additional virtual graphical object, avoid rendering of the additional virtual graphical object via the head mounted display.
12. The artificial reality system of claim 11 , further comprising instructions that, when executed by the at least one processor, cause the artificial reality system to, after rendering the virtual graphical object for display via the head mounted display:
predict, from a second movement of the user of the head mounted display and utilizing the machine learning model, a second future viewpoint of the user; and
based on predicting that the second future viewpoint will not include the location associated with the virtual graphical object, terminate rendering of the virtual graphical object via the head mounted display.
13. The artificial reality system of claim 11 , further comprising instructions that, when executed by the at least one processor, cause the artificial reality system to, after avoiding rendering the additional virtual graphical object for display via the head mounted display:
predict, from a third movement of the user of the head mounted display and utilizing the machine learning model, a third future viewpoint of the user; and
based on predicting that the third future viewpoint will include the location associated with the additional virtual graphical object, render the additional virtual object for display via the head mounted display.
14. The artificial reality system of claim 11 , wherein the movement of the user comprises at least one of a body movement of the user and a head movement of the user.
15. The artificial reality system of claim 11 , further comprising instructions that, when executed by the at least one processor, cause the artificial reality system to determine the movement of the user based on sensor data received from at least one sensor of the head mounted display.
16. A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause a computer system comprising a head mounted display to:
predict, from a movement of a user of the head mounted display and utilizing a machine learning model, a future viewpoint of the user;
based on predicting that the future viewpoint will include a location associated with a virtual graphical object, render the virtual graphical object for display via the head mounted display;
predict, from another movement of the user of the head mounted display and utilizing the machine learning model, an additional future viewpoint of the user; and
based on predicting that the additional future viewpoint will not include an additional location associated with an additional virtual graphical object, avoid rendering of the additional virtual graphical object via the head mounted display.
17. The non-transitory computer-readable medium of claim 16 , wherein:
predicting the future viewpoint will include the location associated with the virtual graphical object is based on a characteristic of the virtual graphical object; and
predicting the additional future viewpoint will not include the additional location associated with the additional graphical object is based on a characteristic of the additional virtual graphical object.
18. The non-transitory computer-readable medium of claim 16 , further comprising instructions that, when executed by the at least one processor, cause the computer system comprising the head mounted display to, after rendering the virtual graphical object for display via the head mounted display:
predict, from a second movement of the user of the head mounted display and utilizing the machine learning model, a second future viewpoint of the user; and
based on predicting that the second future viewpoint will not include the location associated with the virtual graphical object, terminate rendering of the virtual graphical object via the head mounted display.
19. The non-transitory computer-readable medium of claim 16 , further comprising instructions that, when executed by the at least one processor, cause the computer system comprising the head mounted display to, after avoiding rendering the additional virtual graphical object for display via the head mounted display:
predict, from a third movement of the user of the head mounted display and utilizing the machine learning model, a third future viewpoint of the user; and
based on predicting that the third future viewpoint will include the location associated with the additional virtual graphical object, render the additional virtual object for display via the head mounted display.
20. The non-transitory computer-readable medium of claim 16 , further comprising instructions that, when executed by the at least one processor, cause the computer system comprising the head mounted display to predict a future position of the user based on the movement of the user, and wherein the predicting the future viewpoint of the user is further based on the future position.