IP Library Granted Patent US 12,192,072
Granted Patent B2
US 12,192,072 · App. 16/786,565 · Granted Jan 7, 2025

Virtual reality content streaming based on quality of experience level

Inventors: Matthew Ringenberg (Denver, CO); Dhananjay Lal (Englewood, CO); Dell Wolfensparger (Langley, WA); Andrew Ip (Centennial, CO)
Assignee: Charter Communications Operating, LLC
H04L41/5067G02B27/0093G02B27/0172G06T19/006H04L41/5009H04L43/0858H04L47/283H04W28/18G02B2027/0187
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,192,072
App. No.
16/786,565
Granted
Jan 7, 2025
Kind
B2
Abstract

Virtual reality content (VR content) streaming based on quality of experience level is disclosed. In response to a request from a head-mounted display device (HMDD) to serve a VR content, a computing device determines a head motion quantifier (HMQ) associated with the VR content. The computing device is configured to dynamically provide to the HMDD imagery from the VR content based on head movements of a user of the HMDD. The computing device determines a latency value associated with a round-trip time of data between the computing device and the HMDD. The computing device determines a quality of experience level of a plurality of different quality of experience levels based on the HMQ and the latency value, and performs an action based on the quality of experience level.

Claims (81)

1. A method comprising:

receiving, by a computing device, a request from a head-mounted display device (HMDD) to serve first virtual reality (VR) content of a plurality of different VR contents streamable by the computing device, each respective VR content of the plurality of VR contents having a corresponding predetermined head motion quantifier (HMQ), each predetermined HMQ quantifying a rotational velocity of a head of a nominal user participating in the respective VR contents, wherein the computing device is configured to dynamically provide, to the HMDD, imagery from the VR content based on head movements of a user of the HMDD;

obtaining, by the computing device in response to the request, the predetermined HMQ associated with the first VR content;

determining, by the computing device, a latency value associated with a round-trip time of data between the computing device and the HMDD;

determining, by the computing device, a quality of experience level of a plurality of different quality of experience levels based on the predetermined HMQ and the latency value; and

performing, by the computing device, an action based on the quality of experience level.

2. The method of claim 1 wherein the quality of experience level is an undesirable quality of experience level; and

wherein performing the action based on the quality of experience level comprises sending, to the HMDD, a message indicating that the first VR content cannot be provided to the HMDD.

3. The method of claim 1 wherein the quality of experience level is between an undesirable quality of experience level and a desirable quality of experience level; and

wherein performing the action based on the quality of experience level comprises sending, to the HMDD, a message indicating that the first VR content can be provided to the HMDD but will not be a desirable quality of experience.

4. The method of claim 3 wherein sending, to the HMDD, the message indicating that the first VR content can be provided to the HMDD but will not be a desirable quality of experience further comprises:

requesting a confirmation from the HMDD that the first VR content should be provided to the HMDD;

receiving, from the HMDD, the confirmation that the first VR content should be provided to the HMDD; and

in response to receiving the confirmation, begin providing imagery from the first VR content to the HMDD.

5. The method of claim 1 wherein the quality of experience level is a desirable quality of experience level, and wherein performing the action based on the quality of experience level comprises providing the imagery from the first VR content to the HMDD.

6. The method of claim 1 wherein:

the computing device communicates with the HMDD via a communications path that includes one or more other computing devices in a network of computing devices;

wherein the quality of experience level is an undesirable quality of experience level; and

wherein performing the action based on the quality of experience level comprises:

determining that a first computing device of the one or more other computing devices has a latency value associated with a round-trip time of data between the first computing device and the HMDD that would result in a desirable quality of experience level if the first VR content was served from the first computing device; and

transferring the request from the HMDD to serve the first VR content to the first computing device.

7. The method of claim 6 wherein determining that the first computing device of the one or more other computing devices has the latency value associated with the round-trip time of data between the first computing device and the HMDD that would result in the desirable quality of experience level if the first VR content was served from the first computing device further comprises:

sending, by the computing device to the first computing device, a request to provide the latency value associated with the round-trip time of data between the first computing device and the HMDD; and

receiving, from the first computing device, the latency value.

8. The method of claim 6 wherein determining that the first computing device of the one or more other computing devices has the latency value associated with the round-trip time of data between the first computing device and the HMDD that would result in the desirable quality of experience level if the first VR content was served from the first computing device further comprises:

sending, by the computing device to the first computing device and to a second computing device of the one or more other computing devices, a request to provide the latency value associated with the round-trip time of data between the first computing device and the HMDD and a latency value associated with a round-trip time of data between the second computing device and the HMDD, respectively;

receiving a first latency value from the first computing device;

receiving a second latency value from the second computing device;

determining that the first latency value would result in the desirable quality of experience level; and

determining that the second latency value would not result in the desirable quality of experience level.

9. The method of claim 6 wherein determining that the first computing device of the one or more other computing devices has the latency value associated with the round-trip time of data between the first computing device and the HMDD that would result in the desirable quality of experience level if the first VR content was served from the first computing device further comprises:

sending, by the computing device to the first computing device and to a second computing device of the one or more other computing devices, a request to provide the latency value associated with the round-trip time of data between the first computing device and the HMDD and a latency value associated with a round-trip time of data between the second computing device and the HMDD, respectively;

receiving a first latency value from the first computing device;

receiving a second latency value from the second computing device;

determining that the first latency value and the second latency value would result in the desirable quality of experience level;

determining that the first computing device is at a higher hierarchical level in the network of computing devices; and

in response to determining that the first computing device is at a higher hierarchical level in the network of computing devices, transferring the request from the HMDD to serve the first VR content to the first computing device.

10. The method of claim 1 wherein the computing device communicates with the HMDD via a communications path that includes one or more other computing devices in a network of computing devices;

wherein the quality of experience level is an undesirable quality of experience level; and

wherein performing the action based on the quality of experience level comprises:

transferring, to a next computing device in the communications path, the request from the HMDD to serve the first VR content.

11. The method of claim 1 wherein determining, by the computing device, the latency value associated with the round-trip time of data between the computing device and the HMDD comprises:

sending, by the computing device, an internet control message protocol (ICMP) echo request to the HMDD;

receiving, by the computing device from the HMDD, a response to the ICMP echo request; and

determining an amount of time between sending the ICMP echo request and receiving the response to the ICMP echo request.

12. The method of claim 1 wherein obtaining the predetermined HMQ associated with the first VR content comprises:

accessing metadata of the first VR content that identifies the predetermined HMQ associated with the first VR content.

13. A computing device, comprising:

a memory; and

a processor device coupled to the memory configured to:

receive a request from a head-mounted display device (HMDD) to serve first virtual reality (VR) content of a plurality of different VR contents streamable by the computing device, each respective VR content of the plurality of VR contents having a corresponding predetermined head motion quantifier (HMQ), each predetermined HMQ quantifying a rotational velocity of a head of a nominal user participating in the respective VR contents, wherein the computing device is configured to dynamically provide, to the HMDD, imagery from the VR content based on head movements of a user of the HMDD;

obtain, in response to the request, the predetermined HMQ associated with the first VR content;

determine a latency value associated with a round-trip time of data between the computing device and the HMDD;

determine a quality of experience level of a plurality of different quality of experience levels based on the predetermined HMQ and the latency value; and

perform an action based on the quality of experience level.

14. The computing device of claim 13 wherein the quality of experience level is an undesirable quality of experience level; and

wherein to perform the action based on the quality of experience level, the processor device is further configured to send, to the HMDD, a message indicating that the first VR content cannot be provided to the HMDD.

15. The computing device of claim 13 wherein:

the computing device communicates with the HMDD via a communications path that includes one or more other computing devices in the network of computing devices;

wherein the quality of experience level is an undesirable quality of experience level; and

wherein to perform the action based on the quality of experience level, the processor device is further configured to:

determine that a first computing device of the one or more other computing devices has a latency value associated with a round-trip time of data between the first computing device and the HMDD that would result in a desirable quality of experience level if the first VR content was served from the first computing device; and

transfer the request from the HMDD to serve the first VR content to the first computing device.

16. The computing device of claim 13 wherein the computing device communicates with the HMDD via a communications path that includes one or more other computing devices in the network of computing devices;

wherein the quality of experience level is an undesirable quality of experience level; and

wherein to perform the action based on the quality of experience level, the processor device is further configured to:

transfer, to a next computing device in the communications path, the request from the HMDD to serve the first VR content.

17. A computer program product stored on a non-transitory computer-readable storage medium and including instructions configured to cause a processor device to:

receive a request from a head-mounted display device (HMDD) to serve first virtual reality (VR) content of a plurality of different VR contents streamable by the computing device, each respective VR content of the plurality of VR contents having a corresponding predetermined head motion quantifier (HMQ), each predetermined HMQ quantifying a rotational velocity of a head of a nominal user participating in the respective VR contents, wherein the computing device is configured to dynamically provide, to the HMDD, imagery from the VR content based on head movements of a user of the HMDD;

obtain, in response to the request, the predetermined HMQ associated with the first VR content;

determine a latency value associated with a round-trip time of data between the computing device and the HMDD;

determine a quality of experience level of a plurality of different quality of experience levels based on the predetermined HMQ and the latency value; and

perform an action based on the quality of experience level.

18. The computer program product of claim 17 wherein the quality of experience level is an undesirable quality of experience level; and

wherein to perform the action based on the quality of experience level, the instructions further cause the processor device to send, to the HMDD, a message indicating that the first VR content cannot be provided to the HMDD.

19. The computer program product of claim 17 wherein:

the computing device communicates with the HMDD via a communications path that includes one or more other computing devices in the network of computing devices;

wherein the quality of experience level is an undesirable quality of experience level; and

wherein to perform the action based on the quality of experience level, the instructions further cause the processor device to:

determine that a first computing device of the one or more other computing devices has a latency value associated with a round-trip time of data between the first computing device and the HMDD that would result in a desirable quality of experience level if the first VR content was served from the first computing device; and

transfer the request from the HMDD to serve the first VR content to the first computing device.

Assignments (4)
SECURITY INTEREST Recorded Sep 22, 2022
From: CHARTER COMMUNICATIONS OPERATING, LLC; TIME WARNER CABLE ENTERPRISES, LLC
To: WELLS FARGO TRUST COMPANY, N.A.
Reel/Frame 061503/0937 →
SECURITY INTEREST Recorded Sep 22, 2022
From: CHARTER COMMUNICATIONS OPERATING, LLC; TIME WARNER CABLE ENTERPRISES, LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 061504/0307 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Aug 10, 2022
From: CHARTER COMMUNICATIONS OPERATING, LLC; TIME WARNER CABLE ENTERPRISES LLC
To: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 061633/0069 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 12, 2020
From: RINGENBERG, MATTHEW; LAL, DHANANJAY; WOLFENSPARGER, DELL; IP, ANDREW
To: CHARTER COMMUNICATIONS OPERATING, LLC
Reel/Frame 051801/0276 →
Continuity (1)
Related Publication 20210250257A1 · Aug 12, 2021
References Cited (8)
US 20110283011A1 · Li · 2011 [cited by examiner]
US 20170288865A1 · Amidei · 2017 [cited by examiner]
US 20180091629A1 · de la Pena · 2018 [cited by examiner]
US 20190222491A1 · Tomkins · 2019 [cited by examiner]
US 20200037029A1 · He · 2020 [cited by examiner]
US 20200314897A1 · Bendigeri · 2020 [cited by examiner]
US 20200322696A1 · Dvir · 2020 [cited by examiner]
US 20210366204A1 · Thiebaud · 2021 [cited by examiner]