Optimized display image rendering
In one example, a head mounted display system includes at least one memory; and at least one processor to execute instructions to: detect a first position and a first view direction of a head of a user based on sensor data generated by at least one of an accelerometer, at least one camera, or a gyroscope at a first point in time; determine a latency associated with a time to cause an image to be presented on the display; determine a predicted position and a predicted view direction of the head of the user at a second point in time based on the latency; render, prior to the second point in time, the image for presentation on the display based on the predicted position and the predicted view direction of the head of the user; and cause the display to present the rendered image.
1. An apparatus comprising:
network interface circuitry;
memory;
instructions; and
processor circuitry to execute the instructions to:
determine a latency, the latency based on a time to display a rendered image;
determine a future user pose at a prediction interval, wherein the future user pose is based on input data, wherein the input data includes a position and an orientation of a user, and wherein the prediction interval is based on the latency;
render an image for display based on the future user pose; and
cause transmission of the rendered image via the network interface circuitry.
2. The apparatus of claim 1 , wherein the input data is related to motion of a head of the user.
3. The apparatus of claim 1 , wherein the input data includes sensor data from an inertial measurement unit (IMU).
4. The apparatus of claim 1 , wherein the input data includes sensor data from a camera.
5. The apparatus of claim 1 , wherein the latency includes at least one of network latency, render latency, or display latency.
6. The apparatus of claim 1 , wherein the processor circuitry is to determine the future user pose based on Kalman filtering.
7. The apparatus of claim 1 , wherein the processor circuitry is to determine the future user pose based on a series of previous user poses.
8. At least one tangible, non-transitory computer readable storage medium comprising instructions to cause processor circuitry to at least:
determine a latency based on a time to display a rendered image;
determine a future pose of a user at a prediction interval, wherein the future pose is based on input data, the input data includes a position and an orientation of the user, and the prediction interval is based on the latency;
render an image based on the future pose; and
cause transmission of the rendered image to a client device.
9. The at least one computer readable storage medium of claim 8 , wherein the input data is associated with motion of a head of the user.
10. The at least one computer readable storage medium of claim 8 , wherein the input data includes sensor data from an inertial measurement unit (IMU).
11. The at least one computer readable storage medium of claim 8 , wherein the input data includes sensor data from a camera.
12. The at least one computer readable storage medium of claim 8 , wherein the latency includes at least one of network latency, render latency, or display latency.
13. The at least one computer readable storage medium of claim 8 , wherein the instructions cause the processor circuitry to determine the future pose based on Kalman filtering.
14. The at least one computer readable storage medium of claim 8 , wherein the instructions cause the processor circuitry to determine the future pose based on a time series of past poses of the user.
15. An apparatus comprising:
means for connecting via a network;
means for storing to store machine-readable instructions;
means for processing to:
determine a latency to determine a latency based on a time to display a rendered image;
predict a future user pose at a prediction interval, wherein the future user pose is based on input data, wherein the input data includes a position and an orientation of a user, and wherein the prediction interval is based on the latency; and
render an image based on the future user pose; and
means for transmitting to transmit the rendered image for display via the means for connecting.
16. The apparatus of claim 15 , wherein the input data is related to motion of a head of the user.
17. The apparatus of claim 15 , wherein the input data includes sensor data from an inertial measurement unit (IMU).
18. The apparatus of claim 15 , wherein the input data includes sensor data from a camera.
19. The apparatus of claim 15 , wherein the latency includes at least one of network latency, render latency, or display latency.
20. The apparatus of claim 15 , wherein the means for processing is to determine the future user pose based on Kalman filtering.
21. The apparatus of claim 15 , wherein the means for processing is to determine the future user pose based on a time series of past user poses.
22. A method comprising:
determining a latency, the latency based on a time to display a rendered image;
determining, by executing at least one instruction with processor circuitry, a future user pose at a prediction interval, wherein the future user pose is based on input data, the input data includes a position and an orientation of a user, and the prediction interval is based on the latency;
rendering an image based on the future user pose; and
transmitting the rendered image to a device.
23. The method of claim 22 , wherein the input data corresponds to motion of a head of the user.
24. The method of claim 22 , wherein the input data includes sensor data from an inertial measurement unit (IMU).
25. The method of claim 22 , wherein the input data includes sensor data from a camera.
26. The method of claim 22 , wherein the latency includes at least one of network latency, render latency, or display latency.
27. The method of claim 22 , wherein the determining of the future user pose is based on Kalman filtering.
28. The method of claim 22 , wherein the determining of the future user pose is based on previous user poses.