Wireless head mounted display with differential rendering
A method is provided, including: receiving captured images of an interactive environment in which a head-mounted display (HMD) is disposed; receiving inertial data processed from at least one inertial sensor of the HMD; analyzing the captured images of the interactive environment and the inertial data to determine a predicted future location of the HMD; using the predicted future location of the HMD to adjust a beamforming direction of an RF transceiver in a direction that is towards the predicted future location of the HMD; tracking a gaze of a user of the HMD; predicting a movement of the gaze of the user; generating video depicting a view of a virtual environment for the HMD; wherein the regions of the view are rendered differently based on the predicted movement of the gaze of the user; wirelessly transmitting the video via the RF transceiver to the HMD using the adjusted beamforming direction.
1. A method, comprising:
receiving captured images of an interactive environment in which a head-mounted display (HMD) is disposed;
receiving inertial data processed from at least one inertial sensor of the HMD;
analyzing the captured images of the interactive environment and the inertial data to determine a predicted future location of the HMD;
using the predicted future location of the HMD to adjust a beamforming direction of an RF transceiver in a direction that is towards the predicted future location of the HMD;
tracking a gaze of a user of the HMD;
predicting a movement of the gaze of the user to a predicted future region where the user will look next in the virtual environment based on analyzing a trajectory of the gaze of the user and based on analyzing a trajectory of the HMD;
generating video depicting a view of a virtual environment for the HMD;
wherein the regions of the view are rendered differently based on the predicted movement of the gaze of the user, wherein the predicted future region starts being rendered before the gaze of the user is at the predicted future region;
wirelessly transmitting the video via the RF transceiver to the HMD using the adjusted beamforming direction.
2. The method of claim 1 , wherein a region of the view towards which the gaze of the user is directed is rendered at a higher image quality setting than other regions of the view, the other regions of the view being rendered at a lower image quality setting to reduce a size of the video.
3. The method of claim 2 , wherein the image quality setting includes one or more of an update frequency, resolution, complexity of imagery, or a rendering order value that determines an order for rendering the regions of the view.
4. The method of claim 1 , wherein analyzing the captured images and the inertial data includes identifying movement of the HMD, the predicted future location of the HMD being determined using the identified movement of the HMD.
5. The method of claim 4 ,
wherein identifying movement of the HMD includes determining a motion vector of the HMD, the predicted future location of the HMD being determined by applying the motion vector of the HMD to a current location of the HMD;
wherein a magnitude of the motion vector identifies a speed of the movement of the HMD, and wherein a direction of the motion vector identifies a direction of the movement of the HMD.
6. The method of claim 5 , further comprising:
adjusting an angular spread of the RF transceiver based on the speed of the movement of the HMD;
wherein the angular spread increases with increasing speed of the movement of the HMD.
7. The method of claim 1 , wherein the predicted future location of the HMD is determined based on state information of an interactive application that generates the virtual environment.
8. The method of claim 7 , wherein the state information includes data identifying a temporal or geographic location within the virtual environment, such that the predicted future location of the HMD in the interactive environment is determined based on the temporal or geographic location within the virtual environment.
9. The method of claim 1 , wherein the predicted future location of the HMD is determined using a prediction model.
10. The method of claim 9 , wherein the prediction model is improved using a machine learning technique.
11. A method, comprising:
executing an interactive application to generate a virtual environment and render a view of the virtual environment for a head-mounted display (HMD);
wherein rendering the view of the virtual environment includes processing sensor data to determine a location or orientation of the HMD in an interactive environment, and using the location or orientation of the HMD in the interactive environment to determine the view of the virtual environment;
analyzing the sensor data and a state of the interactive application to determine a predicted future location of the HMD, wherein the state of the interactive application is correlated to crowd-sourced HMD location or movement patterns, such that the predicted future location of the HMD in the interactive environment is determined based on the crowd-sourced HMD location or movement patterns;
using the predicted future location of the HMD to adjust a beamforming direction of an RF transceiver in a direction that is towards the predicted future location of the HMD;
wirelessly transmitting video depicting the view of the virtual environment via the RF transceiver to the HMD using the adjusted beamforming direction;
tracking a gaze of a user of the HMD;
predicting a movement of the gaze of the user to a predicted future region where the user will look next in the virtual environment;
wherein the regions of the view are rendered differently based on the predicted movement of the gaze of the user, wherein the predicted future region starts being rendered before the gaze of the user is at the predicted future region.
12. The method of claim 11 , wherein the state of the interactive application includes data identifying a temporal or geographic location within the virtual environment, such that the predicted future location of the HMD in the interactive environment is determined based on the temporal or geographic location within the virtual environment.
13. The method of claim 11 , wherein the predicted future location of the HMD is determined using a prediction model.
14. The method of claim 13 , wherein the prediction model is improved using a machine learning technique.
15. The method of claim 11 , wherein a region of the view towards which the gaze of the user is directed is rendered at a higher image quality setting than other regions of the view, the other regions of the view being rendered at a lower image quality setting to reduce a size of the video.
16. The method of claim 15 , wherein the image quality setting includes one or more of an update frequency, resolution, complexity of imagery, or a rendering order value that determines an order for rendering the regions of the view.
17. The method of claim 11 , wherein the sensor data includes one or more of, captured images of an interactive environment in which a head-mounted display (HMD) is disposed, or inertial data from at least one inertial sensor of the HMD.