Methods and devices for progressive encoding and decoding of multiplane images
Methods and devices for encoding, decoding and transmitting a three-dimensional scene initially represented as a multiplane image (MPI) are provided. Each layer of the MPI is split into patches based on the transparency component. Patches of a layer are grouped in a tile. The greater the depth of the layer, the greater the identifying number of the tile. When several tiles are packed in an atlas image, the same monotonic (i.e. ascending or descending) function according to depth applies to atlas numbers. At the decoding side, the current viewport to render is initially cleared and each decoded tile is sequentially blended over from the nearest one to the furthest due to the numbering of the set of atlases and tiles. Pixels of a patch under rendering are projected onto pixels of the viewport image according to the depth of the tile comprising the patch and metadata indicating the position of the patch in the layer of the MPI the patch has been clustered from.
1 . A method comprising:
obtaining, from a bitstream that indicates volumetric content, a plurality of tiles, wherein each tile is associated with a respective single-layer depth; and
determining, based on the single-layer depths associated with the plurality of tiles, a monotonic order of the plurality of tiles; and
according to the monotonic order:
decoding a first tile of the plurality of tiles;
blending the first tile onto a viewport image to progressively render the volumetric content; and
releasing decoded data associated with the first tile prior to decoding a next tile of the plurality of tiles based on the monotonic order.
2 . The method of claim 1 , further comprising receiving, in the bitstream, an indication that the plurality of tiles are organized for progressive rendering by the monotonic order, wherein decoding of the plurality of tiles is performed according to the monotonic order based on the indication.
3 . The method of claim 1 , wherein the method further comprises:
receiving, in the bitstream, a plurality of atlases, wherein each atlas of the plurality of atlases comprises a respective atlas identifier and one or more tiles of the plurality of tiles, and wherein the atlas identifiers indicate an ordering of the atlases such that tiles of atlases having smaller atlas identifiers are associated with single-layer depths that are closer to the viewport image than tiles of atlases having larger atlas identifiers.
4 . The method of claim 1 , wherein each tile of the plurality of tiles comprises:
a respective tile identifier that corresponds to the single-layer depth associated with the tile, wherein the tile identifiers increase monotonically according to the single-layer depths, and
a respective set of patches, wherein the patches of each tile are grouped based on the single-layer depth associated with the tile.
5 . The method of claim 1 , wherein blending the first tile onto the viewport image comprises projecting and blending color information and transparency information of the first tile onto the viewport image based on the single-layer depth associated with the first tile.
6 . The method of claim 1 , wherein the volumetric content is progressively rendered in a sequential order such that the tiles associated with depth layers nearest to the viewport image are rendered before tiles associated with depth layers farthest from the viewport image.
7 . The method of claim 1 , wherein decoded data for no more than one tile of the plurality of tiles is retained at any time during rendering of the volumetric content.
8 . The method of claim 1 , wherein rendering of the volumetric content is performed while maintaining a bounded decoded-data working set during rendering of the volumetric content.
9 . The method of claim 1 , wherein decoding and rendering of the plurality of tiles is performed prior to receiving all tiles of the plurality of tiles in the bitstream.
10 . A device comprising:
a processor configured to:
obtain, from a bitstream that indicates volumetric content, a plurality of tiles, wherein each tile is associated with a respective single-layer depth;
determine, based on the single-layer depths associated with the plurality of tiles, a monotonic order of the plurality of tiles; and
according to the monotonic order:
decode a first tile of the plurality of tiles;
blend the first tile onto a viewport image to progressively render the volumetric content; and
release decoded data associated with the first tile prior to decoding a next tile of the plurality of tiles based on the monotonic order.
11 . The device of claim 10 , wherein the processor is further configured to receive, in the bitstream, an indication that the plurality of tiles are organized for progressive rendering by the monotonic order, wherein the plurality of tiles are decoded according to the monotonic order based on the indication.
12 . The device of claim 10 , wherein the processor is further configured to:
receive, in the bitstream, a plurality of atlases, wherein each atlas of the plurality of atlases comprises a respective atlas identifier and one or more tiles of the plurality of tiles, and wherein the atlas identifiers indicate an ordering of the atlases such that tiles of atlases having smaller atlas identifiers are associated with single-layer depths that are closer to the viewport image than tiles of atlases having larger atlas identifiers.
13 . The device of claim 10 , wherein each tile of the plurality of tiles comprises:
a respective tile identifier that corresponds to the single-layer depth associated with the tile, wherein the tile identifiers increase monotonically according to the single-layer depths, and
a respective set of patches, wherein the patches of each tile are grouped based on the single-layer depth associated with the tile.
14 . The device of claim 10 , wherein the processor being configured to blend the first tile onto the viewport image comprises the processor being configured to:
project and blend color information and transparency information of the first tile onto the viewport image based on the single-layer depth associated with the first tile.
15 . The device of claim 10 , wherein the volumetric content is progressively rendered in a sequential order such that the tiles associated with depth layers nearest to the viewport image are rendered before tiles associated with depth layers farthest from the viewport image.
16 . The device of claim 10 , wherein decoded data for no more than one tile of the plurality of tiles is retained at any time during rendering of the volumetric content.
17 . The device of claim 10 , wherein the processor is further configured to maintain a bounded decoded-data working set during rendering of the volumetric content.
18 . The device of claim 10 , wherein the processor is further configured to decode and render the plurality of tiles prior to receiving all tiles of the plurality of tiles in the bitstream.