Cross layer traffic optimization for split XR
An improved wireless split rendering system for displaying Extended Reality (XR) content is discussed. A rendering server and client head-mounted device (HMD) may communicate over a wireless medium, where communication control is given to a server application layer logic. This allows the server to use request pose information from the HMD only when needed for rendering, while preserving bandwidth on the wireless medium for transmitting frames of the rendered content. This reduces contention and improves channel efficiency.
1. A method of rendering frames for display to a user at a client device, the method comprising:
initiating a wireless connection with a server, wherein the wireless connection is a WiFi Reverse Direction Grant mode connection and the server is an Access Point (AP) on a WiFi network;
determining a pose information of the user;
responsive to receiving a trigger packet from the server, transmitting the pose information to the server over the wireless connection, wherein the trigger packet indicates permission from the server to transmit during the server's transmit opportunity (TXOP), wherein the server provides permission to transmit during the server's TXOP periodically based on a pose information latency limit required for rendering a frame at the server;
receiving the rendered frame from the server over the wireless connection, the rendered frame rendered for display based on the transmitted pose information; and
displaying the received rendered frame to the user.
2. The method of claim 1 , wherein the transmitted pose information is a 6 degrees of freedom (6 DoF) pose.
3. The method of claim 2 , further comprising:
executing asynchronous time warping (ATW) on the received rendered frame based on a latest pose information before displaying to the user.
4. The method of claim 1 , further comprising:
transmitting the pose information after a predetermined period of time if no trigger packet is received.
5. The method of claim 1 , wherein the rendered frame is a first view of a three-dimensional Virtual Reality (VR) or Augmented Reality (AR) content for display to the user.
6. The method of claim 1 , wherein the trigger packet is transmitted responsive to determining there is no rendered frame to transmit.
7. The method of claim 1 , wherein the trigger packet is transmitted responsive to needing the pose information to render the frame.
8. The method of claim 7 , wherein the trigger packet is initiated by an application layer module executing on the server.
9. An apparatus for rendering frames for display to a user, the apparatus comprising:
a connection processor for initiating a wireless connection with a server and receive a trigger packet, wherein the wireless connection is a WiFi Reverse Direction Grant mode connection and the server is an Access Point (AP) on a WiFi network;
a sensor for determining a pose information of the user;
a processor configured to,
responsive to receiving the trigger packet from the server, transmit the pose information to the server over the wireless connection, wherein the trigger packet indicates permission from the server to transmit during the server's transmit opportunity (TXOP), wherein the server provides permission to transmit during the server's TXOP periodically based on a pose information latency limit required for rendering a frame at the server,
receive the rendered frame from the server over the wireless connection, the rendered frame rendered for display based on the transmitted pose information; and
a display for displaying the received rendered frame to the user.
10. The apparatus of claim 9 , wherein the transmitted pose information is a 6 degrees of freedom (6 DoF) pose.
11. The apparatus of claim 10 , wherein the processor further configured to execute asynchronous time warping (ATW) on the received rendered frame based on a latest pose information before displaying to the user.
12. The apparatus of claim 9 , the processor further configured to transmit the pose information after a predetermined period of time if no trigger packet is received.
13. The apparatus of claim 9 , wherein the rendered frame is a first view of a three-dimensional Virtual Reality (VR) or Augmented Reality (AR) content for display to the user.
14. The apparatus of claim 9 , wherein the trigger packet is transmitted responsive to determining there is no rendered frame to transmit.
15. The apparatus of claim 9 , wherein the trigger packet is transmitted responsive to needing the pose information to render the frame.
16. The apparatus of claim 15 , wherein the trigger packet is initiated by an application layer module executing on the server.
17. The apparatus of claim 9 , wherein the sensor includes at least one of: a gyroscope and an accelerometer.
18. The apparatus of claim 9 , wherein the apparatus is at least one of: head-worn display or a head-mounted display (HMD).
19. A non-transitory computer-readable storage medium storing instructions that, when executed, cause one or more processors to:
initiating a wireless connection with a server, wherein the wireless connection is a WiFi Reverse Direction Grant mode connection and the server is an Access Point (AP) on a WiFi network;
determining a pose information of the user;
responsive to receiving a trigger packet from the server, transmitting the pose information to the server over the wireless connection, wherein the trigger packet indicates permission from the server to transmit during the server's transmit opportunity (TXOP), wherein the server provides permission to transmit during the server's TXOP periodically based on a pose information latency limit required for rendering a frame at the server;
receiving the rendered frame from the server over the wireless connection, the rendered frame rendered for display based on the transmitted pose information; and
displaying the received rendered frame to the user.
20. The medium of claim 19 , wherein the transmitted pose information is a 6 degrees of freedom (6 DoF) pose.
21. The medium of claim 19 , wherein the rendered frame is a first view of a three-dimensional Virtual Reality (VR) or Augmented Reality (AR) content for display to the user.
22. The medium of claim 19 , wherein the trigger packet is transmitted responsive to determining there is no rendered frame to transmit.
23. The medium of claim 19 , wherein the trigger packet is transmitted responsive to needing the pose information to render the frame.
24. The medium of claim 19 , wherein the trigger packet is initiated by an application layer module executing on the server.