Granted Patent B2

US 12,219,115 · App. 18/505,356 · Granted Feb 4, 2025

Inventors: Eyal Ruhm (Tel Aviv, IL); Jill Boyce (Portland, OR); Asaf J. Shenberg (Tel Aviv, IL)

Assignee: Intel Corporation

H04N13/161

Abstract

Claims (44)

1. A apparatus comprising:

one or more processor cores; and

a memory to store data for immersive video;

wherein the one or more processor cores are to:

generate a plurality of patches, each patch representing a portion of video content that is visible from an associated projection direction of a plurality of projection directions for the immersive video;

encode and pack the plurality of patches into a plurality of tiles, each of the plurality of tiles including one or more patches that are visible from an associated projection direction;

identify an immersive video image to be generated, the immersive video image having an associated projection direction and location;

select a set of tiles of the plurality of tiles from the memory for the immersive video image, the set of tiles including tiles for multiple projection directions to be used in generation of the immersive video image;

decode and unpack the patches in the selected set of tiles; and

generate the identified immersive video image, generation of the immersive video image including use of one or more patches from the selected set of tiles.

2. The apparatus of claim 1 , wherein the plurality of patches are generated during a same frame time.

3. The apparatus of claim 1 , wherein the one or more patches encoded and packed in a tile associated with a first projection direction are limited to portions of video content that are occluded from one or more other projection directions of the plurality of projection directions.

4. The apparatus of claim 1 , wherein the one or more processor cores are to decode and unpack the patches from the selected set of tiles without decoding and unpacking one or more other tiles of the plurality of tiles.

5. The apparatus of claim 1 , wherein the one or more processor cores are to select the set of tiles based at least in part on a current or anticipated viewpoint for the immersive video image.

6. The apparatus of claim 1 , wherein the plurality of tiles are included within a coded picture.

7. The apparatus of claim 1 , wherein the plurality of patches includes at least one of texture data and depth data.

8. The apparatus of claim 1 , wherein the plurality of tiles includes motion-constrained tiles in an HEVC (High Efficiency Video Coding) motion-constrained tile set (MCTS).

9. The apparatus of claim 1 , wherein the immersive video includes 6DoF (6 Degrees of Freedom) video.

10. The apparatus of claim 1 , wherein the one or more processor cores includes one or more graphics processor cores of a graphical processing unit (GPU).

11. A non-transitory computer-readable storage medium having stored thereon data representing sequences of instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

generating a plurality of patches, each patch representing a portion of video content that is visible from an associated projection direction of a plurality of projection directions for immersive video;

encoding and packing the plurality of patches into a plurality of tiles, each of the plurality of tiles including one or more patches that are visible from an associated projection direction;

identifying an immersive video image to be generated, the immersive video image having an associated projection direction and location;

selecting a set of tiles of the plurality of tiles for the immersive video image, the set of tiles including tiles for multiple projection directions to be used in generation of the immersive video image;

decoding and unpacking the patches in the selected set of tiles; and

generating the identified immersive video image, generation of the immersive video image including use of one or more patches from the selected set of tiles.

12. The storage medium of claim 11 , wherein the plurality of patches are generated during a same frame time.

13. The storage medium of claim 11 , wherein the one or more patches encoded and packed in a tile associated with a first projection direction are limited to portions of video content that are occluded from one or more other projection directions of the plurality of projection directions.

14. The storage medium of claim 11 , wherein decoding and unpacking the patches from the selected set of tiles includes decoding and unpacking the patches from the selected set of tiles without decoding and unpacking one or more other tiles of the plurality of tiles.

15. The storage medium of claim 11 , wherein selecting the set of tiles includes selecting the set of tiles based at least in part on a current or anticipated viewpoint for the immersive video image.

16. A system comprising:

one or more processors including a graphics processor; and

a memory to store data for immersive video;

wherein the graphics processor is to:

generate a plurality of patches, each patch representing a portion of video content that is visible from an associated projection direction of a plurality of projection directions for the immersive video;

encode and pack the plurality of patches into a plurality of tiles, each of the plurality of tiles including one or more patches that are visible from an associated projection direction;

identify an immersive video image to be generated, the immersive video image having an associated projection direction and location;

select a set of tiles of the plurality of tiles from the memory for the immersive video image, the set of tiles including tiles for multiple projection directions to be used in generation of the immersive video image;

decode and unpack the patches in the selected set of tiles; and

generate the identified immersive video image, generation of the immersive video image including use of one or more patches from the selected set of tiles.

17. The system of claim 16 , wherein the plurality of patches are generated during a same frame time.

18. The system of claim 16 , wherein the one or more patches encoded and packed in a tile associated with a first projection direction are limited to portions of video content that are occluded from one or more other projection directions of the plurality of projection directions.

19. The system of claim 16 , wherein the graphics processor is to decode and unpack the patches from the selected set of tiles without decoding and unpacking one or more other tiles of the plurality of tiles.

20. The system of claim 16 , wherein the graphics processor is to select the set of tiles based at least in part on a current or anticipated viewpoint for the immersive video image.

Continuity (3)

References Cited (251)