Method and apparatus for encoding and decoding three-dimensional scenes in and from a data stream
Methods and devices are provided to encode and decode a data stream carrying data representative of a three-dimensional scene, the data stream comprising color pictures packed in a color image; depth pictures packed in a depth image; and a set of patch data items comprising de-projection data; data for retrieving a color picture in the color image and geometry data. Two types of geometry data are possible. The first type of data describes how to retrieve a depth picture in the depth image. The second type of data comprises an identifier of a parametric function and a list of parameter values for the identified parametric function.
1. A method comprising:
obtaining from a data stream:
a color patch atlas packing color pictures;
a depth patch atlas packing depth pictures corresponding to color patches of a given geometry type; and
metadata comprising, for each color patch of the color patch atlas:
de-projection data;
a color data comprising a description of a location of a color picture in the color image; and
an information indicating whether the color picture is of the given geometry type; and
if the information indicates that the color picture is of the given geometry type, a description of a location of a depth picture in the depth patch atlas; and
if the information indicates that the color picture is not of the given geometry type, a constant depth value; and
for each color patch of the given geometry type:
using de-projection data for de-projecting pixels of the color picture described by the color data at a depth value determined according to pixels of the corresponding depth patch in the depth atlas; and
for each other color patch:
using de-projection data for de-projecting pixels of the color picture described by the color data at the constant depth value in the associated metadata.
2. The method of claim 1 , wherein a color data further comprises a description of at least one of a size or of a shape of at least one color patch in the color patch atlas.
3. The method of claim 1 , wherein the data stream comprises a sequence of color patch atlases, a sequence of depth patch atlases and associated metadata, each depth patch atlas and related metadata being associated with a color patch atlas, wherein the color patch atlases are structured by temporal information.
4. A method comprising:
obtaining a set of color patches, and, for each color patch, an information indicating whether the color patch is of a given geometry type, and if so, an associated depth patch, and if not, an associated constant depth value;
generating a color patch atlas by packing the color patches in a color patch atlas and generating a color data comprising a description of a location of each color patch as packed in the color patch atlas;
generating a depth patch atlas by packing depth patches associated with color patches of the given geometry type and generating first metadata comprising a description of a location of the depth patch in the depth patch atlas; and generating second metadata comprising the associated constant depth value for other color patches; and
encoding in a data stream:
the generated color patch atlas;
the generated depth patch atlas; and
for each color patch, the information indicating whether the color picture is of the given geometry type, and, if so the first metadata, and if not, the second metadata.
5. The method of claim 4 , wherein a color data further comprises a description of at least one of a size or of a shape of at least one color patch in the color patch atlas.
6. The method of claim 4 , wherein the data stream comprises a sequence of color patch atlases, a sequence of depth patch atlases and associated metadata, each depth patch atlas and related metadata being associated with a color patch atlas, wherein the color patch atlases are structured by temporal information.
7. A device comprising electronic circuitry adapted for:
obtaining from a data stream:
a color patch atlas packing color pictures;
a depth patch atlas packing depth pictures corresponding to color patches of a given geometry type; and
metadata comprising, for each color patch of the color patch atlas:
de-projection data;
a color data comprising a description of a location of a color picture in the color image; and
an information indicating whether the color picture is of the given geometry type; and
if the information indicates that the color picture is of the given geometry type, a description of a location of a depth picture in the depth patch atlas; and
if the information indicates that the color picture is not of the given geometry type, a constant depth value; and
for each color patch of the given geometry type:
using de-projection data for de-projecting pixels of the color picture described by the color data at a depth value determined according to pixels of the corresponding depth patch in the depth atlas; and
for each other color patch:
using de-projection data for de-projecting pixels of the color picture described by the color data at the constant depth value in the associated metadata.
8. The device of claim 7 , wherein a color data further comprises a description of at least one of a size or of a shape of at least one color patch in the color patch atlas.
9. The device of claim 7 , wherein the data stream comprises a sequence of color patch atlases, a sequence of depth patch atlases and associated metadata, each depth patch atlas and related metadata being associated with a color patch atlas, wherein the color patch atlases are structured by temporal information.
10. A device comprising electronic circuitry adapted for:
obtaining a set of color patches, and, for each color patch, an information indicating whether the color patch is of a given geometry type, and if so, an associated depth patch, and if not, an associated constant depth value;
generating a color patch atlas by packing the color patches in a color patch atlas and generating a color data comprising a description of a location of each color patch as packed in the color patch atlas;
generating a depth patch atlas by packing depth patches associated with color patches of the given geometry type and generating first metadata comprising a description of a location of the depth patch in the depth patch atlas; and generating second metadata comprising the associated constant depth value for other color patches; and
encoding in a data stream:
the generated color patch atlas;
the generated depth patch atlas; and
for each color patch, the information indicating whether the color picture is of the given geometry type, and, if so the first metadata, and if not, the second metadata.
11. The device of claim 10 , wherein a color data further comprises a description of at least one of a size or of a shape of at least one color patch in the color patch atlas.
12. The device of claim 10 , wherein the data stream comprises a sequence of color patch atlases, a sequence of depth patch atlases and associated metadata, each depth patch atlas and related metadata being associated with a color patch atlas, wherein the color patch atlases are structured by temporal information.
13. A non-transitory computer readable medium having stored instructions which, when executed by one or more processors, cause the one or more processors to carry out the method of claim 1 .
14. A non-transitory computer readable medium having stored instructions which, when executed by one or more processors, cause the one or more processors to carry out the method of claim 4 .