IP Library Granted Patent US 11,503,289
Granted Patent B2
US 11,503,289 · App. 17/314,296 · Granted Nov 15, 2022

Bitstream structure for viewport-based streaming with a fallback bitstream

Inventors: Rohit Abhishek (San Jose, CA); Byeongdoo Choi (Palo Alto, CA); Shuai Zhao (Pleasanton, CA); Stephan Wenger (Hillsborough, CA)
Assignee: TENCENT AMERICA LLC
H04N19/119H04N19/115H04N19/162H04N19/177H04N19/184
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,503,289
App. No.
17/314,296
Granted
Nov 15, 2022
Kind
B2
Abstract

A method, computer program, and computer system is provided for splitting viewport bitstreams. A first coded video bitstream is at least partially encoded using a first Group-of-Pictures (GOP) structure at a first resolution. A second coded video bitstream structured into a plurality of tiles is encoded using a second GOP structure at a second resolution, whereby the second GOP structure includes fewer coded pictures than the first GOP structure. A streaming bitstream for decoding or rendering is created using the first coded video bitstream and the second coded bitstream.

Claims (53)

1. A method of splitting viewport bitstreams, executable by a processor, comprising:

encoding, at least partially, a first coded video bitstream using a first Group-of-Pictures (GOP) structure at a first resolution;

encoding a second coded video bitstream structured into a plurality of tiles using a second GOP structure at a second resolution, wherein the second GOP structure comprises fewer coded pictures than the first GOP structure;

creating a streaming bitstream for decoding or rendering using the first coded video bitstream and the second coded video bitstream, wherein the streaming bitstream comprises a greater number of random access points than the first coded video bitstream; and

transmitting the first coded video bitstream at an actual bitrate that fits a target bitrate in response to a slow change of a user's field of view (FoV).

2. The method of claim 1 , wherein the second coded video bitstream has a shorter segment size than the first coded video bitstream.

3. The method of claim 2 , wherein the second coded video bitstream is associated with the user's field of view (FoV).

4. The method of claim 3 , further comprising:

transmitting the first coded video bitstream at the actual bitrate that fits the target bitrate in response to insufficient network bandwidth.

5. The method of claim 3 , further comprising:

transmitting the second coded video bitstream in response to a quick change of the user's FoV.

6. The method of claim 1 , further comprising

offering for delivery, over Dynamic Adaptive Streaming over HTTP (DASH), at least a one first segment of the first coded video bitstream and a plurality of second segments of the second coded video bitstream,

wherein at least one of the second segments includes at least one tile of the second video bitstream but less than the whole second video bitstream.

7. The method of claim 1 , further comprising:

upsampling a first picture; and

blending the first picture with the second picture in response to the first coded video bitstream and the second coded video bitstream being transmitted and decoded at a decoder,

wherein a second picture of the second coded video bitstream covers a part or a whole region of a first picture of the first coded video bitstream.

8. The method of claim 7 , wherein blending the first picture with the second picture further comprises:

replacing or averaging the part or the whole region of the first picture by the second picture.

9. A computer system for splitting viewport bitstreams, the computer system comprising:

one or more computer-readable non-transitory storage media configured to store computer program code; and

one or more computer processors configured to access said computer program code and operate as instructed by said computer program code, said computer program code including:

first encoding code configured to cause the one or more computer processors to encode, at least partially, a first coded video bitstream using a first Group-of-Pictures (GOP) structure at a first resolution;

second encoding code configured to cause the one or more computer processors to encode a second coded video bitstream structured into a plurality of tiles using a second GOP structure at a second resolution, wherein the second GOP structure comprises fewer coded pictures than the first GOP structure;

creating code configured to cause the one or more computer processors to create a streaming bitstream for decoding or rendering using the first coded video bitstream and the second coded video bitstream, wherein the streaming bitstream comprises a greater number of random access points than the first coded video bitstream; and

first transmitting code configured to cause the one or more computer processors to transmit the first coded video bitstream at an actual bitrate that fits a target bitrate in response to a slow change of a user's field of view (FoV).

10. The computer system of claim 9 , wherein the second resolution is higher than the first resolution.

11. The computer system of claim 10 , wherein the second coded video bitstream has a shorter segment size than the first coded video bitstream.

12. The computer system of claim 11 , wherein the second coded video bitstream is associated with the user's field of view (FoV).

13. The computer system of claim 12 , further comprising:

second transmitting code configured to cause the one or more computer processors to transmit the first coded video bitstream at the actual bitrate that fits the target bitrate based on insufficient network bandwidth.

14. The computer system of claim 12 , further comprising:

third transmitting code configured to cause the one or more computer processors to transmit the second coded video bitstream based on a quick change of the user's FoV.

15. The computer system of claim 9 , wherein the streaming bitstream is structured with a reduced random-access point period.

16. The computer system of claim 9 , further comprising:

offering code configured to cause the one or more computer processors to offer for delivery, over Dynamic Adaptive Streaming over HTTP (DASH), at least a one first segment of the first coded video bitstream and a plurality of second segments of the second coded video bitstream,

wherein at least one of the second segments includes at least one tile of the second video bitstream but less than the whole second video bitstream.

17. The computer system of claim 9 , further comprising:

upsampling code configured to cause the one or more computer processors to upsample a first picture; and

blending code configured to cause the one or more computer processors to blend the first picture with the second picture based on the first coded video bitstream and the second coded video bitstream being transmitted and decoded at a decoder,

wherein a second picture of the second coded video bitstream covers a part or a whole region of a first picture of the first coded video bitstream.

18. The computer system of claim 17 , wherein the blending code further comprises:

respective replacing and averaging code configured to cause the one or more computer processors to replace or average the part or the whole region of the first picture by the second picture.

19. A non-transitory computer readable medium having stored thereon a computer program for splitting viewport bitstreams, the computer program configured to cause one or more computer processors to:

encode, at least partially, a first coded video bitstream using a first Group-of-Pictures (GOP) structure at a first resolution;

encode a second coded video bitstream structured into a plurality of tiles using a second GOP structure at a second resolution, wherein the second GOP structure comprises fewer coded pictures than the first GOP structure;

create a streaming bitstream for decoding or rendering using the first coded video bitstream and the second coded video bitstream, wherein the streaming bitstream comprises a greater number of random access points than the first coded video bitstream; and

transmit the first coded video bitstream at an actual bitrate that fits a target bitrate in response to a slow change of a user's field of view (FoV).

20. The computer readable medium of claim 19 , wherein the computer program is further configured to cause the one or more computer processors to:

transmit the second coded video bitstream based on a quick change of the user's FoV,

wherein the second resolution is higher than the first resolution, wherein the second coded video bitstream has a shorter segment size than the first coded video bitstream, and

wherein the second coded video bitstream is associated with the user's field of view (FoV).

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 7, 2021
From: ABHISHEK, ROHIT; CHOI, BYEONGDOO; ZHAO, SHUAI; WENGER, STEPHAN
To: TENCENT AMERICA LLC
Reel/Frame 056170/0023 →
Continuity (2)
Provisional Application 63030856 · May 27, 2020
Related Publication 20210377527A1 · Dec 2, 2021