IP Library Granted Patent US 11,711,415
Granted Patent B2
US 11,711,415 · App. 17/374,565 · Granted Jul 25, 2023

Measuring quality-of-experience (QoE) for virtual reality (VR) streaming content

Inventors: Matthew Ringenberg (Denver, CO); Dhananjay Lal (Englewood, CO); Dell Wolfensparger (Langley, WA); Andrew Ip (Centennial, CO)
Assignee: Charter Communications Operating, LLC
H04L65/80G06F3/012H04L43/087H04L43/0829H04L65/70H04N13/327
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,711,415
App. No.
17/374,565
Granted
Jul 25, 2023
Kind
B2
Abstract

Measuring quality-of-experience (QoE) for virtual reality (VR) streaming content is disclosed. A network computing device receives a client-side VR stream capture and a client pose data set that are generated by a client computing device based on a VR content and one or more induced network impairments (e.g., latency, packet loss, and/or jitter, as non-limiting examples). Using the same VR content and the client pose data set, the network computing device generates a source VR stream capture that is not subjected to the one or more induced network impairments. The network computing device performs a frame-by-frame comparison of the client-side VR stream capture and the source VR stream capture. Based on the frame-by-frame comparison, the network computing device generates a QoE metric that indicates a degree of degradation of the client-side VR stream capture relative to the source VR stream capture.

Claims (49)

1. A method, comprising:

streaming, by a client computing device via a network connection to a virtual reality (VR) head-mounted display device (HMDD) communicatively coupled to the client computing device, VR content received from a VR server of a network computing device;

recording, by the client computing device, movements of the HMDD based on the VR content while streaming the VR content from the VR server to the HMDD and selectively rotating the HMDD by a computer-controlled turntable to which the HMDD is affixed as a client pose data set;

recording, by the client computing device, visual output of the HMDD based on the VR content as a client-side VR stream capture; and

transmitting, by the client computing device, the client-side VR stream capture and the client pose data set to the network computing device.

2. The method of claim 1 , wherein the HMDD is selectively rotated by the computer-controlled turntable according to a rotation-and-velocity script.

3. The method of claim 1 , wherein the client pose data set comprises data indicating one or more of a position of the HMDD and an orientation of the HMDD.

4. The method of claim 1 , further comprising injecting, by the network computing device, one or more induced network impairments into the network connection while streaming the VR content;

wherein:

recording the movements of the HMDD is further based on the one or more induced network impairments; and

recording the visual output of the HMDD is further based on the one or more induced network impairments.

5. The method of claim 4 , wherein the one or more induced network impairments comprises one or more of network latency, packet loss, and jitter.

6. The method of claim 5 , further comprising:

receiving, by the network computing device from the client computing device, the client-side VR stream capture and the client pose data set;

generating a source VR stream capture based on the VR content and the client pose data set without the one or more induced network impairments;

performing a frame-by-frame comparison of the client-side VR stream capture and the source VR stream capture; and

generating a quality-of-experience (QoE) metric indicating a degree of degradation of the client-side VR stream capture relative to the source VR stream capture, based on the frame-by-frame comparison.

7. The method of claim 6 , wherein the QoE metric comprises a Structural Similarity Index Metric (SSIM).

8. The method of claim 6 , further comprising identifying, based on the QoE metric, a network location from which to stream the VR content.

9. A computing system, comprising:

a client computing device comprising:

a first memory; and

a first processor device coupled to the first memory and configured to:

stream, via a network connection to a virtual reality (VR) head-mounted display device (HMDD) communicatively coupled to the client computing device, VR content received from a VR server of a network computing device;

record movements of the HMDD based on the VR content while streaming the VR content from the VR server to the HMDD and selectively rotating the HMDD by a computer-controlled turntable to which the HMDD is affixed as a client pose data set;

record visual output of the HMDD based on the VR content as a client-side VR stream capture; and

transmit the client-side VR stream capture and the client pose data set to the network computing device.

10. The computing system of claim 9 , wherein the HMDD is selectively rotated by the computer-controlled turntable according to a rotation-and-velocity script.

11. The computing system of claim 9 , wherein the client pose data set comprises data indicating one or more of a position of the HMDD and an orientation of the HMDD.

12. The computing system of claim 9 , further comprising:

the network computing device comprising:

a second memory; and

a second processor device coupled to the second memory and configured to inject one or more induced network impairments into the network connection while streaming the VR content;

wherein:

the first processor device is configured to record the movements of the HMDD further based on the one or more induced network impairments; and

the first processor device is configured to record the visual output of the HMDD further based on the one or more induced network impairments.

13. The computing system of claim 12 , wherein the one or more induced network impairments comprises one or more of network latency, packet loss, and jitter.

14. The computing system of claim 12 , wherein the second processor device is further configured to:

receive, from the client computing device, the client-side VR stream capture and the client pose data set;

generate a source VR stream capture based on the VR content and the client pose data set without the one or more induced network impairments;

perform a frame-by-frame comparison of the client-side VR stream capture and the source VR stream capture; and

generate a quality-of-experience (QoE) metric indicating a degree of degradation of the client-side VR stream capture relative to the source VR stream capture, based on the frame-by-frame comparison.

15. The computing system of claim 14 , wherein the QoE metric comprises a Structural Similarity Index Metric (SSIM).

16. The computing system of claim 14 , wherein the second processor device is further configured to identify, based on the QoE metric, a network location from which to stream the VR content.

17. A computer program product stored on a non-transitory computer-readable storage medium and including computer-executable instructions configured to cause a processor device to:

stream, via a network connection to a virtual reality (VR) head-mounted display device (HMDD) communicatively coupled to a client computing device, VR content received from a VR server of a network computing device;

record movements of the HMDD based on the VR content while streaming the VR content from the VR server to the HMDD and selectively rotating the HMDD by a computer-controlled turntable to which the HMDD is affixed as a client pose data set;

record visual output of the HMDD based on the VR content as a client-side VR stream capture; and

transmit the client-side VR stream capture and the client pose data set to the network computing device.

Assignments (4)
SECURITY INTEREST Recorded Sep 22, 2022
From: CHARTER COMMUNICATIONS OPERATING, LLC; TIME WARNER CABLE ENTERPRISES, LLC
To: WELLS FARGO TRUST COMPANY, N.A.
Reel/Frame 061503/0937 →
SECURITY INTEREST Recorded Sep 22, 2022
From: CHARTER COMMUNICATIONS OPERATING, LLC; TIME WARNER CABLE ENTERPRISES, LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 061504/0307 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Aug 10, 2022
From: CHARTER COMMUNICATIONS OPERATING, LLC; TIME WARNER CABLE ENTERPRISES LLC
To: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 061633/0069 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 13, 2021
From: RINGENBERG, MATTHEW; LAL, DHANANJAY; WOLFENSPARGER, DELL; IP, ANDREW
To: CHARTER COMMUNICATIONS OPERATING, LLC
Reel/Frame 056841/0326 →
Continuity (2)
Continuation 16743805 · Jan 15, 2020
Related Publication 20210344736A1 · Nov 4, 2021