Backchannel resilience for virtual, augmented, or mixed reality (xR) applications in connectivity-constrained environments
Embodiments of systems and methods for backchannel resilience in virtual, augmented, or mixed reality (xR) applications in connectivity-constrained environments are described. In an embodiment, a Head-Mounted Device (HMD) may include a processor and a memory coupled to the processor, the memory having program instructions stored thereon that, upon execution by the processor, cause the HMD to: encode each of a plurality of Simultaneous Localization and Mapping (SLAM) landmarks, in part, based upon the SLAM landmark's distance to the HMD; and transmit the encoded SLAM landmarks to an Information Handling System (IHS) coupled to the HMD.
1. A Head-Mounted Device (HMD), comprising:
a processor; and
a memory coupled to the processor, the memory having program instructions stored thereon that, upon execution by the processor, cause the HMD to:
receive a Look-Up Table (LUT) from an Information Handling System (IHS) coupled to the HMD, wherein the LUT comprises: a threshold distance, a first resilience parameter, and a second resilience parameter;
encode each of a plurality of Simultaneous Localization and Mapping (SLAM) landmarks; and
at least one of:
transmit a first encoded SLAM landmark to the IHS using the first resilience parameter in response to a first distance between the first SLAM landmark and the HMD being smaller than the threshold distance; or
transmit a second encoded SLAM landmark to the IHS using the second resilience parameter in response to a second distance between the second SLAM landmark and the HMD being greater than the threshold distance.
2. The HMD of claim 1 , further comprising Infra-Red (IR) or Near-IR (NIR) camera configured to provide IR or NIR image frames to the processor.
3. The HMD of claim 1 , wherein the plurality of SLAM landmarks are selected to have a Kalman Gain above a threshold value.
4. The HMD of claim 1 , wherein the resilience parameter further comprises at least one of: a transmission protocol or a packet re-transmission setting.
5. The HMD of claim 1 , wherein the first resilience parameter indicates a Transmission Control Protocol (TCP) protocol, and wherein the second parameter indicates a User Datagram Protocol (UDP) protocol.
6. The HMD of claim 1 , wherein the first resilience parameter indicates a User Datagram Protocol (UDP) protocol, and wherein the second parameter indicates a Transmission Control Protocol (TCP) protocol.
7. The HMD of claim 1 , wherein the IHS is configured to transmit another LUT to the HMD, or modify the LUT, periodically or based upon context information.
8. The HMD of claim 7 , wherein the context information comprises a type of xR application being executed.
9. The HMD of claim 7 , wherein the context information comprises a gaze vector.
10. The HMD of claim 7 , wherein the context information comprises data loss statistics.
11. The HMD of claim 7 , wherein the context information comprises detection of a safety event at a distance smaller or greater than the threshold distance.
12. The HMD of claim 1 , wherein the IHS is configured to:
decode the encoded SLAM landmarks; and
produce a map using the decoded SLAM landmarks.
13. A method, comprising:
receiving a Look-Up Table (LUT) from an Information Handling System (IHS) coupled to a Head-Mounted Device (HMD), wherein the LUT comprises: a threshold distance, a first resilience parameter, and a second resilience parameter,
encoding, by the HMD, each of a plurality of Simultaneous Localization and Mapping (SLAM) landmarks; and
at least one of:
transmitting a first encoded SLAM landmark to the IHS using the first resilience parameter in response to a first distance between the first SLAM landmark and the HMD being smaller than the threshold distance; or
transmitting a second encoded SLAM landmark to the IHS using the second resilience parameter in response to a second distance between the second SLAM landmark and the HMD being greater than the threshold distance.
14. The method of claim 13 , further comprising selecting the LUT among a plurality of LUTs based upon at least one of: a type of xR application, a gaze vector, detection of a safety event, or a data loss metric.
15. A hardware memory of a Head-Mounted Device (HMD) wearable by a user, the hardware memory having program instructions stored thereon that, upon execution by a processor, cause the HMD to:
receive a Look-Up Table (LUT) from an Information Handling System (IHS) coupled to the HMD, wherein the LUT comprises: a threshold distance, a first resilience parameter, and a second resilience parameter;
encode each of a plurality of Simultaneous Localization and Mapping (SLAM) landmarks; and
at least one of:
transmit a first encoded SLAM landmark to the IHS using the first resilience parameter in response to a first distance between the first SLAM landmark and the HMD being smaller than the threshold distance; or
transmit a second encoded SLAM landmark to the IHS using the second resilience parameter in response to a second distance between the second SLAM landmark and the HMD being greater than the threshold distance.
16. The hardware memory of claim 15 , wherein one of the first or second resilience parameters enables packet re-transmission, and the other of the first or second resilience parameters disables packet re-transmission.