IP Library › Granted Patent US 10,846,042
Granted Patent B2
US 10,846,042 · App. 16/176,276 · Granted Nov 24, 2020

Adaptive rendering for untethered multi-user virtual reality

Inventors: Christina Vlachou (Palo Alto, CA); Xing Liu (Palo Alto, CA); Kyu-Han Kim (Palo Alto, CA)
Assignee: Hewlett Packard Enterprise Development LP
G06F3/1431G06T9/00G06T15/04H04W28/20G06F3/011H04W84/12
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,846,042
App. No.
16/176,276
Granted
Nov 24, 2020
Kind
B2
Abstract

Example method includes: rendering, by a network device, a plurality of frames for a multi-user Virtual Reality (VR) environment at a first quality and a second quality, storing, at the network device, the plurality of frames at the first quality and the second quality, transmitting, from the network device and to an untethered mobile device, a first frame of the plurality of frames at the first quality using Wi-Fi, determining, by the network device, that an available bandwidth to the untethered mobile device does not satisfy a threshold, and transmitting, by the network device and in response to determining, a subsequent frame at the second quality using Wi-Fi.

Claims (55)

1. A method comprising:

dividing visual elements into static elements and dynamic objects;

pre-rendering, by a network device, a plurality of frames including one or more of the static elements and the dynamic objects for a multi-user Virtual Reality (VR) environment at a first quality and a second quality to generate presentable image renderings of each of the plurality of frames in accordance with a 360 degree view from each possible user position, the pre-rendered plurality of frames at the first quality and the pre-rendered plurality of frames at the second quality being simultaneously available for transmission, wherein the second quality is a lower quality than the first quality, and wherein the pre-rendered frames consist of the static elements that are not changeable by users in the multi-user VR environment, and the dynamic objects, the pre-rendering of which comports with a machine learning model-predicted polygon complexity;

storing, at the network device, at least one of compressed and encrypted versions of the pre-rendered plurality of frames at the first quality and the second quality;

transmitting, from the network device and to an untethered mobile device, a first frame of the stored pre-rendered plurality of frames at the first quality using Wi-Fi;

determining, by the network device, that an available Wi-Fi bandwidth to the untethered mobile device does not satisfy a threshold; and

in response to determining that the available bandwidth does not satisfy the threshold, transmitting, by the network device, a subsequent frame of the stored pre-rendered plurality of frames at the second quality using Wi-Fi.

2. The method of claim 1 , wherein the network device transmits the first frame and the subsequent frame to the untethered mobile device through a wireless access point.

3. The method of claim 1 , further comprising:

performing the method for each of a plurality of untethered mobile devices participating in the multi-user VR environment, including transmitting frames with different qualities to a first untethered mobile device and a second untethered mobile device.

4. The method of claim 1 , further comprising:

receiving, from the untethered mobile device, a position in the multi-user VR environment; and

transmitting the first frame based on the position.

5. The method of claim 1 , further comprising:

determining, based on the available Wi-Fi bandwidth, that the predicted polygon complexity of the one or more dynamic objects should be reduced; and

in response to determining, transmitting a lower predicted polygon complexity for the one or more dynamic objects to the untethered mobile device.

6. The method of claim 1 , further comprising:

receiving, from a wireless access point associated with the untethered mobile device, the available Wi-Fi bandwidth to the untethered mobile device.

7. A network device comprising at least:

a memory;

a central processing unit (CPU) executing instructions from the memory to:

divide visual elements into static elements and dynamic objects;

pre-render a plurality of frames including one or more of the static elements and the dynamic objects for a multi-user Virtual Reality (VR) environment at a first quality and a second quality to generate presentable image renderings of each of the plurality of frames in accordance with a 360 degree view from each possible user position, the pre-rendered plurality of frames at the first quality and the pre-rendered plurality of frames at the second quality being simultaneously available for transmission, wherein the second quality is a lower quality than the first quality, and wherein the pre-rendered frames consist of the static elements that are not changeable by users in the multi-user VR environment, the static elements including background, and the dynamic objects, the pre-rendering of which comports with a machine learning model-predicted polygon complexity;

store at least one of compressed and encrypted versions of the pre-rendered plurality of frames at the first quality and the second quality;

transmit, to an untethered mobile device, a first frame of the stored pre-rendered plurality of frames at the first quality using Wi-Fi;

determine that an available Wi-Fi bandwidth to the untethered mobile device does not satisfy a threshold; and

in response to determining that the available bandwidth does not satisfy the threshold, transmit a subsequent frame of the stored pre-rendered plurality of frames at the second quality using Wi-Fi.

8. The network device of claim 7 , wherein the network device transmits the first frame and the subsequent frame to the untethered mobile device through a wireless access point.

9. The network device of claim 7 , the instructions further configured to:

perform the method for each of a plurality of untethered mobile devices participating in the multi-user VR environment, including transmitting frames with different qualities to a first untethered mobile device and a second untethered mobile device of the plurality of untethered mobile devices.

10. The network device of claim 7 , the instructions further configured to:

receive, from the untethered mobile device, a position in the multi-user VR environment; and

transmit the first frame based on the position.

11. The network device of claim 7 , the instructions further configured to:

determine, based on the available Wi-Fi bandwidth, that the predicted polygon complexity of one or more of the dynamic objects should be reduced; and

in response to determining, transmit a lower predicted polygon complexity for the one or more dynamic object to the untethered mobile device.

12. The network device of claim 7 , the instructions further configured to:

receive, from a wireless access point associated with the untethered mobile device, the available Wi-Fi bandwidth to the untethered mobile device.

13. A non-transitory computer-readable storage medium encoded with instructions executable by a main processor of central processing unit (CPU), the CPU comprising at least the main processor and a plurality of co-processors, the non-transitory machine-readable storage medium comprising instructions to:

divide visual elements into static elements and dynamic objects

pre-render a plurality of frames including one or more of the static elements and the dynamic objects for a multi-user Virtual Reality (VR) environment at a first quality and a second quality to generate presentable image renderings of each of the plurality of frames in accordance with a 360 degree view from each possible user position, the pre-rendered plurality of frames at the first quality and the pre-rendered plurality of frames at the second quality being simultaneously available for transmission, wherein the second quality is a lower quality than the first quality, and wherein the pre-rendered frames consist of the static elements that are not changeable by users in the multi-user VR environment, the static elements including background, and the dynamic objects, the pre-rendering of which comports with a machine learning model-predicted polygon complexity;

store at least one of compressed and encrypted versions of the pre-rendered plurality of frames at the first quality and the second quality;

transmit, to an untethered mobile device, a first frame of the stored pre-rendered plurality of frames at the first quality using Wi-Fi;

determine that an available Wi-Fi bandwidth to the untethered mobile device does not satisfy a threshold; and

in response to determining that the available bandwidth does not satisfy the threshold, transmit a subsequent frame of the stored pre-rendered plurality of frames at the second quality using Wi-Fi.

14. The non-transitory computer-readable storage medium of claim 13 , wherein the first frame and the subsequent frame are transmitted to the untethered mobile device through a wireless access point.

15. The non-transitory computer-readable storage medium of claim 13 , the instructions further configured to:

perform the method for each of a plurality of untethered mobile devices participating in the multi-user VR environment, including transmitting frames with different qualities to a first untethered mobile device and a second untethered mobile device of the plurality of untethered mobile devices.

16. The non-transitory computer-readable storage medium of claim 13 , the instructions further configured to:

receive, from the untethered mobile device, a position in the multi-user VR environment; and

transmit the first frame based on the position.

17. The non-transitory computer-readable storage medium of claim 13 , the instructions further configured to:

determine, based on the available Wi-Fi bandwidth, that the predicted polygon complexity of the one or more dynamic objects should be reduced; and

in response to determining, transmit a lower predicted polygon complexity for the one or more dynamic objects to the untethered mobile device.

18. The method of claim 1 , wherein the static elements include background elements.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 18, 2019
From: VLACHOU, CHRISTINA; LIU, XING; KIM, KYU-HAN
To: HEWLETT PACKARD ENTERPRISE DEVELOPMENT LP
Reel/Frame 050762/0594 →
Continuity (1)
Related Publication 20200133610A1 · Apr 30, 2020