IP Library Granted Patent US 11,195,498
Granted Patent B2
US 11,195,498 · App. 16/743,765 · Granted Dec 7, 2021

Compensating for latency in a streaming virtual reality environment

Inventors: Dhananjay Lal (Englewood, CO); Andrew Ip (Centennial, CO); Matthew Ringenberg (Denver, CO); Dell Wolfensparger (Langley, WA); Mun G. Choi (Denver, CO)
Assignee: Charter Communications Operating, LLC
G09G5/37G06F3/012G09G5/14H04L43/0864G09G2354/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,195,498
App. No.
16/743,765
Granted
Dec 7, 2021
Kind
B2
Abstract

Compensating for latency in a streaming virtual reality environment is disclosed. A computing device receives from a head-mounted display device (HMDD) having a field of view (FOV), first pose data that identifies an orientation of the HMDD worn by a participant viewing a virtual world via the HMDD. The computing device renders, based at least in part on the first pose data and a first oversized FOV that is greater than the FOV, oversized imagery that depicts a portion of the virtual world within the FOV of the HMDD and a portion of the virtual world outside of the FOV of the HMDD. The computing device communicates the oversized imagery to the HMDD.

Claims (61)

1. A method comprising:

receiving, by a computing device, a request to stream a first virtual world of a plurality of different virtual worlds streamable by the computing device, each respective virtual world of the plurality of virtual worlds having a corresponding predetermined head movement quantifier of a plurality of predetermined head movement quantifiers, each predetermined head movement quantifier quantifying a rotational velocity of a head movement for the respective virtual world;

accessing, by the computing device, a first predetermined head movement quantifier of the plurality of predetermined head movement quantifiers identified for the first virtual world;

receiving, by the computing device from a head-mounted display device (HMDD) having a field of view (FOV), first pose data that identifies an orientation of the HMDD worn by a participant viewing the first virtual world via the HMDD;

determining a first oversized FOV that is greater than the FOV based at least in part on the first predetermined head movement quantifier;

rendering, based at least in part on the first pose data and the first oversized FOV, oversized imagery that depicts a portion of the first virtual world within the FOV of the HMDD and a portion of the first virtual world outside of the FOV of the HMDD; and

communicating the oversized imagery to the HMDD.

2. The method of claim 1 wherein determining the first oversized FOV based at least in part on the first predetermined head movement quantifier further comprises:

determining a round-trip time (RTT) value based on an elapsed time of a signal communicated from the computing device through at least one intermediate switch device to the HMDD and receipt, by the computing device, of a response from the HMDD to the signal; and

determining the first oversized FOV based at least in part on the first predetermined head movement quantifier and on the RTT value.

3. The method of claim 2 wherein determining the RTT value comprises:

sending, by the computing device, an internet control message protocol (ICMP) echo request to the HMDD;

receiving, by the computing device from the HMDD, a response to the ICMP echo request; and

determining an amount of time between sending the ICMP echo request and receiving the response to the ICMP echo request.

4. The method of claim 3 wherein determining the RTT value further comprises adding a predetermined value to the amount of time between sending the ICMP echo request and receiving the response to the ICMP echo request to derive the RTT value.

5. The method of claim 2 wherein determining the RTT value comprises accessing a predetermined RTT value associated with the first virtual world.

6. The method of claim 1 wherein the first predetermined head movement quantifier is based on historical measurements of rotational velocities of head movements of users interacting with the first virtual world.

7. The method of claim 1 further comprising:

receiving second pose data that identifies the orientation of the HMDD worn by the participant viewing the first virtual world;

determining a round-trip time (RTT) value based on an elapsed time of a signal communicated from the computing device through at least one intermediate switch device to the HMDD and receipt, by the computing device, of a response from the HMDD to the signal;

based on the RTT value, determining a second oversized FOV that is different from the first oversized FOV;

rendering, based at least in part on the second pose data and the second oversized FOV, oversized imagery that depicts a portion of the first virtual world within the FOV of the HMDD and a portion of the first virtual world outside of the FOV of the HMDD; and

communicating the oversized imagery to the HMDD.

8. The method of claim 1 further comprising:

receiving, by the computing device from the HMDD, second pose data that identifies an orientation of the HMDD worn by the participant viewing a second virtual world via the HMDD, the second virtual world being a different virtual world than the first virtual world;

accessing a second predetermined head movement quantifier identified for the second virtual world, the second predetermined head movement quantifier being different from the first predetermined head movement quantifier;

determining a second oversized FOV that is greater than the FOV based at least in part on the second predetermined head movement quantifier;

rendering, based at least in part on the second pose data and the second oversized FOV, second virtual world oversized imagery that depicts a portion of the second virtual world within the FOV of the HMDD and a portion of the second virtual world outside of the FOV of the HMDD; and

communicating the second virtual world oversized imagery to the HMDD.

9. A computing device, comprising:

a memory; and

a processor device coupled to the memory configured to:

receive a request to stream a first virtual world of a plurality of different virtual worlds streamable by the computing device, each respective virtual world of the plurality of virtual worlds having a corresponding predetermined head movement quantifier of a plurality of predetermined head movement quantifiers, each predetermined head movement quantifier quantifying a rotational velocity of a head movement for the respective virtual world;

access a first predetermined head movement quantifier of the plurality of predetermined head movement quantifiers identified for the first virtual world;

receive, from a head-mounted display device (HMDD) having a field of view (FOV), first pose data that identifies an orientation of the HMDD worn by a participant viewing the first virtual world via the HMDD;

determine a first oversized FOV that is greater than the FOV based at least in part on the first predetermined head movement quantifier;

render, based at least in part on the first pose data and the first oversized FOV, oversized imagery that depicts a portion of the first virtual world within the FOV of the HMDD and a portion of the first virtual world outside of the FOV of the HMDD; and

communicate the oversized imagery to the HMDD.

10. The computing device of claim 9 wherein the processor device is further configured to:

determine a round-trip time (RTT) value based on an elapsed time of a signal communicated from the computing device through at least one intermediate switch device to the HMDD and receipt, by the computing device, of a response from the HMDD to the signal; and

determine the first oversized FOV based at least in part on the first predetermined head movement quantifier and on the RTT value.

11. The computing device of claim 10 wherein to determine the RTT value the processor device is further configured to access a predetermined RTT value associated with the first virtual world.

12. The computing device of claim 10 wherein to determine the RTT value the processor device is further configured to:

send, by the computing device, an internet control message protocol (ICMP) echo request to the HMDD;

receive, by the computing device from the HMDD, a response to the ICMP echo request; and

determine an amount of time between sending the ICMP echo request and receiving the response to the ICMP echo request.

13. The computing device of claim 9 wherein the first predetermined head movement quantifier is based on historical measurements of rotational velocities of head movements of users interacting with the first virtual world.

14. A computer program product stored on a non-transitory computer-readable storage medium and including instructions configured to cause a processor device to:

receive a request to stream a first virtual world of a plurality of different virtual worlds streamable by the processor device, each respective virtual world of the plurality of virtual worlds having a corresponding predetermined head movement quantifier of a plurality of predetermined head movement quantifiers, each predetermined head movement quantifier quantifying a rotational velocity of a head movement for the respective virtual world;

access a first predetermined head movement quantifier of the plurality of predetermined head movement quantifiers identified for the first virtual world;

receive, from a head-mounted display device (HMDD) having a field of view (FOV), first pose data that identifies an orientation of the HMDD worn by a participant viewing the first virtual world via the HMDD;

determine a first oversized FOV that is greater than the FOV based at least in part on the first predetermined head movement quantifier;

render, based at least in part on the first pose data and the first oversized FOV, oversized imagery that depicts a portion of the first virtual world within the FOV of the HMDD and a portion of the first virtual world outside of the FOV of the HMDD; and

communicate the oversized imagery to the HMDD.

15. The computer program product of claim 14 wherein the first predetermined head movement quantifier is based on historical measurements of rotational velocities of head movements of users interacting with the first virtual world.

16. The computer program product of claim 14 wherein the instructions further cause the processor device to:

receive second pose data that identifies the orientation of the HMDD worn by the participant viewing the first virtual world;

determine a round-trip time (RTT) value based on an elapsed time of a signal communicated from the processor device through at least one intermediate switch device to the HMDD and receipt, by the processor device, of a response from the HMDD to the signal;

based on the RTT value, determine a second oversized FOV that is different from the first oversized FOV;

render, based at least in part on the second pose data and the second oversized FOV, oversized imagery that depicts a portion of the first virtual world within the FOV of the HMDD and a portion of the virtual world outside of the FOV of the HMDD; and

communicate the oversized imagery to the HMDD.

Assignments (4)
SECURITY INTEREST Recorded Sep 22, 2022
From: CHARTER COMMUNICATIONS OPERATING, LLC; TIME WARNER CABLE ENTERPRISES, LLC
To: WELLS FARGO TRUST COMPANY, N.A.
Reel/Frame 061503/0937 →
SECURITY INTEREST Recorded Sep 22, 2022
From: CHARTER COMMUNICATIONS OPERATING, LLC; TIME WARNER CABLE ENTERPRISES, LLC
To: THE BANK OF NEW YORK MELLON TRUST COMPANY, N.A.
Reel/Frame 061504/0307 →
SUPPLEMENTAL SECURITY AGREEMENT Recorded Aug 10, 2022
From: CHARTER COMMUNICATIONS OPERATING, LLC; TIME WARNER CABLE ENTERPRISES LLC
To: BANK OF AMERICA, N.A., AS ADMINISTRATIVE AGENT
Reel/Frame 061633/0069 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 22, 2020
From: LAL, DHANANJAY; IP, ANDREW; RINGENBERG, MATTHEW; WOLFENSPARGER, DELL; CHOI, MUN G.
To: CHARTER COMMUNICATIONS OPERATING, LLC
Reel/Frame 051587/0222 →
Continuity (1)
Related Publication 20210217386A1 · Jul 15, 2021