APPARATUS AND METHODS FOR ADAPTIVE BIT-RATE STREAMING OF 360 VIDEO
In some aspects, the disclosure is directed to methods and systems for providing 360-degree video. One or more viewport engines receives 360-degree video data, and generates a plurality of versions of the 360-degree video data. Each version has a corresponding primary viewing direction, independently decodable units (IDUs) of video data arranged according to a sequence, and image resolution for the corresponding primary viewing direction that is higher than image resolutions for other viewing directions. A streaming server streams a first IDU of video content from a first version of the plurality of versions of the 360-degree video data to a client for rendering at the client, and receives feedback according to the rendering at the client. Responsive to the update in viewing direction, the streaming server dynamically switches to a second version of the plurality of versions that has a primary viewing direction comprising the second viewing direction.
1 . A system for streaming 360-degree video data, comprising:
one or more viewport engines configured to:
receive 360-degree video data; and
generate a plurality of versions of the 360-degree video data, each version having a corresponding primary viewing direction, each version comprising independently decodable units (IDUs) of video data arranged according to a sequence, and having image resolution for the corresponding primary viewing direction that is higher than image resolutions for other viewing directions; and
a streaming server configured to:
stream a first IDU of video content from a first version of the plurality of versions of the 360-degree video data to a client for rendering at the client, the first version having a primary viewing direction comprising a first viewing direction;
receive feedback according to the rendering at the client, the feedback indicative of an update in viewing direction to a second viewing direction; and
responsive to the update in viewing direction, dynamically switch to a second version of the plurality of versions that has a primary viewing direction comprising the second viewing direction, to stream one or more IDUs from the second version to the client, the one or more IDUs streamed according to the sequence relative to the first IDU.
2 . The system of claim 1 , wherein the one or more viewport engines are configured to generate IDUs of the first and second versions for streaming at a first bit rate.
3 . The system of claim 2 , wherein the first bit rate is a function of an image resolution and a frame rate of the IDUs of the first and second versions.
4 . The system of claim 1 , wherein the one or more viewport engines are configured to generate the plurality of versions of the 360-degree video data by generating a first plurality of versions according to a first bit rate for streaming, and a second plurality of versions according to a second bit rate for streaming.
5 . The system of claim 4 , wherein the streaming server is further configured to detect a change in network condition, and to dynamically switch from the first version to the second version responsive to the detected change in network condition, wherein the first version is from the first plurality of versions, and the second version is from the second plurality of versions.
6 . The system of claim 4 , wherein the streaming server is further configured to detect a change in network condition, and to dynamically switch from the second version to a third version, responsive to the detected change in network condition, wherein the second version is from the first plurality of versions, and the third version is from the second plurality of versions, and the second and third versions each has a primary viewing direction comprising the second viewing direction.
7 . The system of claim 1 , wherein each IDU of the first version comprises video data for which a typical field-of-view (FOV) angle centered around the corresponding primary viewing direction has a first image resolution, and video data for other viewing directions each has an image resolution that is lower than the first image resolution.
8 . The system of claim 1 , further comprising one or more storage devices configured to store the generated plurality of versions of the 360-degree video data, wherein the streaming server is further configured to retrieve the first IDU from the one or more storage devices to stream to the client.
9 . A method for streaming 360-degree video data, comprising:
receiving 360-degree video data;
generating a plurality of versions of the 360-degree video data, each version having a corresponding primary viewing direction, each version comprising independently decodable units (IDUs) of video data arranged according to a sequence, and having image resolution for the corresponding primary viewing direction that is higher than image resolutions for other viewing directions;
streaming, by a streaming server, a first IDU of video content from a first version of the plurality of versions of the 360-degree video data to a client for rendering at the client, the first version having a primary viewing direction comprising a first viewing direction;
receiving, by the streaming server, feedback according to the rendering at the client, the feedback indicative of an update in viewing direction to a second viewing direction; and
responsive to the update in viewing direction, dynamically switching to a second version of the plurality of versions that has a primary viewing direction comprising the second viewing direction, to stream one or more IDUs from the second version to the client, the one or more IDUs streamed according to the sequence relative to the first IDU.
10 . The method of claim 9 , further comprising generating IDUs of the first and second versions for streaming at a first bit rate.
11 . The method of claim 10 , wherein the first bit rate is a function of an image resolution and a frame rate of the IDUs of the first and second versions.
12 . The method of claim 9 , wherein generating the plurality of versions of the 360-degree video data comprises generating a first plurality of versions according to a first bit rate for streaming, and a second plurality of versions according to a second bit rate for streaming.
13 . The method of claim 12 , further comprising detecting a change in network condition, and dynamically switching from the first version to the second version responsive to the detected change in network condition, wherein the first version is from the first plurality of versions, and the second version is from the second plurality of versions.
14 . The method of claim 12 , further comprising detecting a change in network condition, and dynamically switching from the second version to a third version, responsive to the detected change in network condition, wherein the second version is from the first plurality of versions, and the third version is from the second plurality of versions, and the second and third versions each has a primary viewing direction comprising the second viewing direction.
15 . The method of claim 9 , wherein each IDU of the first version comprises video data for which a typical field-of-view (FOV) angle centered around the corresponding primary viewing direction has a first image resolution, and video data for other viewing directions each has an image resolution that is lower than the first image resolution.
16 . The method of claim 9 , further comprising storing the generated plurality of versions of the 360-degree video data, and retrieving the first IDU from the one or more storage devices to stream to the client.
17 . One or more computer-readable storage media having instructions stored therein that, when executed by at least one processor, cause the at least one processor to perform operations comprising:
receiving 360-degree video data;
generating a plurality of versions of the 360-degree video data, each version having a corresponding primary viewing direction, each version comprising independently decodable units (IDUs) of video data arranged according to a sequence, and having image resolution for the corresponding primary viewing direction that is higher than image resolutions for other viewing directions;
streaming a first IDU of video content from a first version of the plurality of versions of the 360-degree video data to a client for rendering at the client, the first version having a corresponding primary viewing direction comprising a first viewing direction;
receiving feedback according to the rendering at the client, the feedback indicative of an update in viewing direction to a second viewing direction; and
responsive to the update in viewing direction, dynamically switching to a second version of the plurality of versions that has a primary viewing direction comprising the second viewing direction, to stream one or more IDUs from the second version to the client, the one or more IDUs streamed according to the sequence relative to the first IDU.
18 . The one or more computer-readable storage media of claim 17 , wherein the at least one processor further performs operations comprising generating IDUs of the first and second versions for streaming at a first bit rate.
19 . The one or more computer-readable storage media of claim 18 , wherein the first bit rate is a function of an image resolution and a frame rate of the IDUs of the first and second versions.
20 . The one or more computer-readable storage media of claim 17 , wherein generating the plurality of versions of the 360-degree video data comprises generating a first plurality of versions according to a first bit rate for streaming, and a second plurality of versions according to a second bit rate for streaming.