Prioritizing tile-based virtual reality video streaming using adaptive rate allocation

1. In a digital medium environment for providing a virtual reality experience, a computer-implemented method of streaming bandwidth-efficient virtual reality videos, the method comprising:

partitioning a three-dimensional (3D) sphere mesh into a plurality of 3D mesh segments that comprises a 3D top spherical cap, a 3D bottom spherical cap, and a set of 3D middle mesh segments;

partitioning, based on the plurality of 3D mesh segments, a spherical video into a plurality of 3D tiles that comprises a 3D top spherical cap tile, a 3D bottom spherical cap tile, and a set of 3D middle tiles;

determining an available bandwidth to stream the plurality of 3D tiles;

assigning the plurality of 3D tiles to priority levels based on a distance from each 3D tile to a viewport location of a client device; and

streaming the plurality of 3D tiles to the client device by streaming:

3D tiles at a given priority level at a same bit rate,

3D tiles at a lowest priority level at, at least, a minimum bit rate to provide a predetermined minimum video quality, and

3D tiles at a highest priority level at a maximum bit rate determined by a difference between the available bandwidth and a minimum bandwidth required to stream the plurality of 3D tiles at the minimum bit rate.

2. The method as recited in claim 1 , further comprising partitioning the spherical video by dividing the set of 3D middle tiles into four equal sized 3D spherical cap tiles.

3. The method as recited in claim 2 , wherein assigning the plurality of 3D tiles to a priority level based on a distance from each 3D tile to the viewport location of the client device comprises assigning the plurality of 3D tiles to at least the lowest priority level, a middle priority level, and the highest priority level.

4. The method as recited in claim 3 , further comprising streaming 3D tiles at the middle priority level at a next maximum bit rate determined by a difference between the available bandwidth and a bandwidth required to stream the 3D tiles at the lowest priority level at the minimum bit rate and the 3D tiles at the highest priority level at the maximum bit rate.

5. The method as recited in claim 3 , wherein assigning the plurality of 3D tiles to a priority level based on a distance from each 3D tile to the viewport location of the client device comprises assigning each 3D tile that abuts a 3D tile assigned to the highest priority level the middle priority level and assigning remaining 3D tiles to the lowest priority level.

6. The method as recited in claim 1 , wherein streaming the 3D tiles assigned to each priority level to the client device comprises allocating bit rates to the priority levels using a multiple-choice knapsack problem algorithm.

7. The method as recited in claim 1 , further comprising:

updating assignment of the plurality of 3D tiles to the priority levels upon a change in the viewport location; and

adaptively allocating bit rates to the 3D tiles at the priority levels based on the updating of the assignment of the plurality of 3D tiles to the priority levels.

8. A system for streaming virtual reality videos, the system comprising:

one or more computer-readable memories comprising a spherical video; and

at least one computing device configured to cause the system to:

receive viewport information from a client device comprising a current viewport location;

partition a three-dimensional (3D) sphere mesh into a plurality of 3D mesh segments that comprises a 3D top spherical cap, a 3D bottom spherical cap, and a set of 3D middle mesh segments;

partition, based on the plurality of 3D mesh segments, a spherical video into a plurality of 3D tiles by dividing the spherical video into a 3D top spherical cap tile, a 3D bottom spherical cap tile, and a set of four 3D middle tiles;

determine an available bandwidth to stream the plurality of 3D tiles;

assign the plurality of 3D tiles to priority levels based on a distance from each tile to the current viewport location; and

stream the plurality of 3D tiles to the client device by streaming:

3D tiles at a lowest priority level at, at least, a minimum bit rate to provide a predetermined minimum video quality, and

3D tiles at a highest priority level at a maximum bit rate determined by a difference between the available bandwidth and a minimum bandwidth required to stream the plurality of 3D tiles at the minimum bit rate.

9. The system of claim 8 , wherein the at least one computing device is further configured to cause the system to assign the plurality of 3D tiles to priority levels based on a distance from each 3D tile to the current viewport location by:

identify one or more 3D tiles of the plurality of 3D tiles that overlap with the current viewport location;

identify one or more 3D tiles of the plurality of 3D tiles that abut against the one or more 3D tiles of the plurality of 3D tiles that overlap with the current viewport location;

assign the highest priority level to the one or more 3D tiles that overlap with the current viewport location;

assign a next highest priority level to the one or more 3D tiles that abut against the one or more 3D tiles of the plurality of 3D tiles that overlap with the current viewport location; and

assign the lowest priority level to a remaining one or more 3D tiles of the plurality of D3 tiles.

10. The system of claim 9 , wherein the at least one computing device is further configured to cause the system to:

detect an object in at least one 3D tile from the one or more 3D tiles that abut against the one or more 3D tiles of the plurality of 3D tiles that overlap with the current viewport location; and

based on detecting the object, reassigning the at least one 3D tile from the next highest priority level to the highest priority level.

11. The system of claim 8 , wherein the at least one computing device is further configured to cause the system to stream 3D tiles at a given priority level at a same bit rate.

12. The system of claim 8 , wherein the at least one computing device is further configured to cause the system to:

update assignment of the plurality of 3D tiles to the priority levels upon a change in the current viewport location; and

adaptively allocate bit rates to the 3D tiles at the priority levels based on the updating of the assignment of the plurality of 3D tiles to the priority levels.

13. The system of claim 8 , wherein:

the client device comprises a head-mounted display with a viewport size being between 90-degrees and 96-degrees; and

the at least one computing device is configured to partition the set of four 3D middle 3D tiles into four 3D middle tiles between 90-degrees and 96-degrees without portioning the 3D top spherical cap.

14. A system for streaming virtual reality videos, the system comprising:

one or more computer-readable memories comprising a panorama video; and

at least one computing device configured to cause the system to:

receive a current viewport location of a client device;

partition a three-dimensional (3D) sphere mesh into a plurality of 3D mesh segments that comprises a 3D top spherical cap, a 3D bottom spherical cap, and a set of 3D middle mesh segments;

partition, based on the plurality of 3D mesh segments, a panorama video into a plurality of 3D tiles by dividing the panorama video into a hexaface sphere comprising a 3D top spherical cap tile, a 3D bottom spherical cap tile, and a set of four 3D middle tiles;

determine an available bandwidth to stream the plurality of 3D tiles;

assign the plurality of 3D tiles to priority levels based on a distance from each 3D tile to the current viewport location; and

stream the plurality of 3D tiles to the client device by streaming:

3D tiles at a lowest priority level at, at least, a minimum bit rate to provide a predetermined minimum video quality, and

3D tiles at a highest priority level at a maximum bit rate determined by a difference between the available bandwidth and a minimum bandwidth required to stream the plurality of 3D tiles at the minimum bit rate.

15. The system as recited in claim 14 , wherein the panorama video comprises a 360-degree virtual reality video and the client device comprises a head mounted display.

16. The system as recited in claim 14 , wherein the at least one computing device is further configured to cause the system to assign the plurality of 3D tiles to a priority level based on a distance from each 3D tile to the current viewport location of the client device by assigning each 3D tile that at least partially overlaps the current viewport location the highest priority level.

17. The system as recited in claim 16 , wherein the at least one computing device is further configured to cause the system to assign the plurality of 3D tiles to a priority level based on a distance from each 3D tile to the current viewport location of the client device by assigning the plurality of 3D tiles to at least the lowest priority level, a middle priority level, and the highest priority level.

18. The system as recited in claim 17 , wherein the at least one computing device is further configured to cause the system to stream 3D tiles at the middle priority level at a next maximum bit rate determined by a difference between the available bandwidth and a bandwidth required to stream the 3D tiles at the lowest priority level at the minimum bit rate and the 3D tiles at the highest priority level at the maximum bit rate.

19. The system as recited in claim 14 , wherein the at least one computing device is further configured to cause the system to assign the plurality of 3D tiles to a priority level based on a distance from each 3D tile to the current viewport location of the client device by assigning each 3D tile that abuts a 3D tile assigned to the highest priority level a middle priority level and assigning remaining 3D tiles to the lowest priority level.

20. The system as recited in claim 14 , wherein the at least one computing device is further configured to cause the system to:

update assignment of the plurality of 3D tiles to the priority levels upon a change in the current viewport location; and

adaptively allocate bit rates to the 3D tiles at the priority levels based on the updating of the assignment of the plurality of 3D tiles to the priority levels.