IP Library › Granted Patent US 11,259,050
Granted Patent B2
US 11,259,050 · App. 17/019,591 · Granted Feb 22, 2022

Six degrees of freedom spatial layout signaling

Inventors: Lauri Ilola (Tampere, FI); Kimmo Roimela (Tampere, FI); Vinod Kumar Malamal Vadakital (Tampere, FI); Jaakko Keranen (Helsinki, FI)
Assignee: Nokia Technologies Oy
H04N19/597H04N13/161H04N19/142H04N19/147
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,259,050
App. No.
17/019,591
Granted
Feb 22, 2022
Kind
B2
Abstract

Apparatuses, methods, and computer programs are disclosed to implement six degrees of freedom spatial layout signaling. An example apparatus includes at least one processor; and at least one non-transitory memory including computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to perform: segment a scene of volumetric video into one or more sequences, each sequence comprising one or more sub viewing volumes having one or more bitstream alternatives; generate viewing volume and view point indications within one or more dynamic adaptive streaming over hypertext transfer protocol media presentation description files; generate one or more sub volume scheme uniform resource identifiers for client side instructions to retrieve content; and provide one or more of the sub viewing volumes based on a client selection and request.

Claims (68)

1. An apparatus comprising:

at least one processor; and

at least one non-transitory memory including computer program code;

wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to:

segment a scene of volumetric video into one or more sequences, each sequence comprising one or more sub viewing volumes having one or more bitstream alternatives;

generate viewing volume and view point indications within one or more dynamic adaptive streaming over hypertext transfer protocol media presentation description files;

generate one or more sub volume scheme uniform resource identifiers for client side instructions to retrieve content;

provide one or more of the sub viewing volumes based on a client selection and request; and

hierarchically arrange viewing volumes based on signaling of parent sub viewing volumes.

2. The apparatus of claim 1 , wherein centroids of the sub viewing volumes form a grid or tetrahedral mesh for selecting preferred sub viewing volumes for any viewing point within a larger viewing volume.

3. The apparatus of claim 1 , wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to:

define data structures for dynamic adaptive streaming over hypertext transfer protocol, wherein the data structures comprise a three-dimensional point in space, a set of shapes used to generate the sub viewing volumes, a number of groups of shapes, and an operator primitive to combine the shapes and the groups of shapes.

4. The apparatus of claim 1 , wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to:

signal information about centroids of the sub viewing volumes to the adaptation sets using one or more viewpoint attributes.

5. The apparatus of claim 1 , wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to:

divide the media presentation description files into smaller media presentation description files;

provide signaling information about the sub viewing volumes, wherein the signaling information about the sub viewing volumes is provided using at least one of one or more levels of the media presentation description files, or one or more periods of the media presentation description files;

receive a request for a media presentation description file based on a viewing position of a client;

select a partial viewing volume containing the requested media presentation description file; and

provide the selected partial viewing volume to the client.

6. The apparatus of claim 1 , wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to:

implement an index file listing uniform resource locators for the sub viewing volume of the media presentation description files.

7. The apparatus of claim 1 , wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to:

implement a uniform resource identifier scheme to associate a segmented static sub viewing volume with a particular uniform resource locator.

8. The apparatus of claim 1 , wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to:

dynamically create the media presentation description files based on a client device viewing position.

9. The apparatus of claim 1 , wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to:

signal a viewing orientation of a sub viewing volume.

10. The apparatus of claim 1 , wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to:

link sub viewing volumes by exit direction.

11. An apparatus comprising:

at least one processor; and

at least one non-transitory memory including computer program code;

wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to:

monitor a viewing position of a client device within a viewing volume level of one or more media presentation description files;

request new viewing volume data when approaching one or more edges of a level of at least one of the media presentation description files;

select one or more sub volume atlas streams from a media presentation description file based on the viewing position and an orientation of the client device; and

render a novel view from the selected one or more sub volume atlas streams, wherein the rendering comprises combining individual immersive video streams as a union of currently streamed sub viewing volumes of a scene.

12. The apparatus of claim 11 , wherein the viewing volume data has been segmented into one or more sequences, each sequence comprising one or more sub viewing volumes having one or more bitstream alternatives.

13. The apparatus of claim 11 , wherein centroids of the sub viewing volumes form a grid or tetrahedral mesh for selecting preferred sub viewing volumes for any viewing point within a larger viewing volume.

14. The apparatus of claim 11 , wherein data structures for dynamic adaptive streaming over hypertext transfer protocol are defined, wherein the data structures comprise a three-dimensional point in space, a set of shapes used to generate the sub viewing volumes, a number of groups of shapes, and an operator primitive to combine the shapes and the groups of shapes.

15. The apparatus of claim 11 , wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to:

download a media presentation description file from a server, and selecting a desired sub viewing volume based on information about centroids in the media presentation description file.

16. The apparatus of claim 11 , wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to:

request the media presentation description file based on the viewing position of the client device;

select a partial viewing volume containing the requested media presentation description file based on a client viewing point; and

receive the selected partial viewing volume from a server;

wherein the media presentation description files have been divided into smaller media presentation description files.

17. The apparatus of claim 11 , wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to:

implement an index file listing uniform resource locators for the sub viewing volumes of the media presentation description file.

18. The apparatus of claim 11 , wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to:

request a segmented static sub viewing volume based on an associated uniform resource identifier.

19. The apparatus of claim 11 , wherein the media presentation description files are dynamically created based on the viewing position of the client device.

20. The apparatus of claim 11 , wherein viewing volumes are hierarchically arranged based on signaling of parent sub viewing volumes.

21. The apparatus of claim 11 , wherein a viewing orientation of a sub viewing volume is signaled.

22. The apparatus of claim 11 , wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus at least to:

request the media presentation description file for a next sub viewing volume in an exit direction.

23. A method comprising:

segmenting a scene of volumetric video into one or more sequences, each sequence comprising one or more sub viewing volumes having one or more bitstream alternatives;

generating viewing volume and view point indications within one or more dynamic adaptive streaming over hypertext transfer protocol media presentation description files;

generating one or more sub volume scheme uniform resource identifiers for client side instructions to retrieve content;

providing one or more of the sub viewing volumes based on a client selection and request; and

signaling information about centroids of the sub viewing volumes to the adaptation sets using one or more viewpoint attributes.

24. A method comprising:

monitoring a viewing position of a client device within a viewing volume level of one or more media presentation description files;

requesting new viewing volume data when approaching one or more edges of a level of at least one of the media presentation description files;

selecting one or more sub volume atlas streams from a media presentation description file based on the viewing position and an orientation of the client device; and

rendering a novel view from the selected one or more sub volume atlas streams, wherein the rendering comprises combining individual immersive video streams as a union of currently streamed sub viewing volumes of a scene.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 16, 2020
From: ILOLA, LAURI; ROIMELA, KIMMO; VADAKITAL, VINOD KUMAR MALAMAL; KERANEN, JAAKKO
To: NOKIA TECHNOLOGIES OY
Reel/Frame 053790/0131 →
Continuity (2)
Provisional Application 62902519 · Sep 19, 2019
Related Publication 20210092444A1 · Mar 25, 2021