Method and apparatus for encoding and decoding three-dimensional scenes in and from a data stream
Methods and devices are provided to encode and decode a data stream carrying data representative of a three-dimensional scene, the data stream comprising color pictures packed in a color image; depth pictures packed in a depth image; and a set of patch data items comprising de-projection data; data for retrieving a color picture in the color image and geometry data. Two types of geometry data are possible. The first type of data describes how to retrieve a depth picture in the depth image. The second type of data comprises an identifier of a parametric function and a list of parameter values for the identified parametric function.
1. A method of decoding a three-dimensional scene from a data stream, the method comprising:
obtaining from said data stream:
color pictures packed in a color image;
depth pictures packed in a depth image; and
metadata comprising, for each color picture of the color image:
de-projection data;
a color data comprising a description of a location of a color picture in the color image; and
a geometry data of a type belonging to a group of types comprising:
a first geometry type comprising a description of a location of a depth picture in the depth image; and
a second geometry type comprising an identifier of a parametric function and a list of parameter values for the identified parametric function; and
for each color picture of the color image:
on condition that the geometry data is of the first geometry type, using de-projection data for de-projecting pixels of the color picture described by the color data at a depth value determined according to pixels of the depth picture described by the geometry data; and
on condition that the geometry data is of the second geometry type, using de-projection data for de-projecting pixels of the color picture described by the color data at a depth value determined by the parametric function of the geometry data, the parametric function initialized with the list of parameter values.
2. The method of claim 1 , wherein the color data further comprises a description of a size and/or of a shape of the color picture in the color image.
3. The method of claim 1 , wherein the identifier of the parametric function points to a function of a list of parametric functions stored in memory.
4. The method of claim 1 , wherein the stream comprises a sequence of color images, depth images and associated metadata, a depth image and related metadata being associated with a color image, the color images being structured by temporal information.
5. A method for encoding a three-dimensional scene in a data stream, the method comprising:
obtaining a set of patches, each patch comprising:
de-projection data;
a color picture; and
geometry data comprising:
a first geometry type comprising a depth picture; or
a second geometry type comprising an identifier of a parametric function and a list of parameter values for the identified parametric function;
generating a color image by packing color pictures of the patches and a color data comprising a description of a location of the color picture as packed in the color image;
generating a depth image by packing depth pictures of patches having a geometry data of the first geometry type and replacing the depth picture in the geometry data of corresponding patch by a description of a location of the depth picture of the patch as packed in the depth image; and
encoding in the stream:
the generated color image;
the generated depth image; and
metadata comprising the de-projection data, the color data, and the geometry data of every patch.
6. The method of claim 5 , wherein the color data further comprises a description of a size and/or of a shape of the color picture in the color image.
7. The method of claim 5 , wherein the identifier of the parametric function points to a function of a list of parametric functions stored in memory.
8. The method of claim 5 , wherein the stream comprises a sequence of color images, depth images and associated metadata, a depth image and related metadata being associated with a color image, the color images being structured by temporal information.
9. A non-transitory processor readable medium storing data representative of a three-dimensional scene, the data stream comprising:
at least a color image comprising color pictures packed in the color image;
at least a depth image comprising depth pictures packed in the depth image, a depth image being associated with a first color image; and
metadata associated with each color picture of the color image comprising:
de-projection data;
a color data comprising a description of a location of a color picture in the first color image; and
a geometry data comprising:
a description of a location of a depth picture in the depth image associated with the first color image; or
an identifier of a parametric function and a list of parameter values for the identified parametric function;
wherein the data stream comprises at least one color picture with geometry data of a first geometry type and at least one color picture with geometry data of a second geometry type.
10. The non-transitory processor readable medium of claim 9 , wherein the color data further comprises a description of a size and/or of a shape of the color picture in the color image.
11. A device for decoding a three-dimensional scene from a data stream, the device comprising a memory associated with a processor configured to:
obtain from the data stream:
color pictures packed in the color image;
depth pictures packed in the depth image; and
metadata comprising, for each color picture of the color image:
de-projection data;
a color data comprising a description of a location of a color picture in the color image; and
a geometry data of a type belonging to a group of types comprising:
a first geometry type comprising a description of a location of a depth picture in the depth image; and
a second geometry type comprising an identifier of a parametric function and a list of parameter values for the identified parametric function; and
for each color picture of the color image, determine to:
on condition that the geometry data is of the first geometry type, use de-projection data for de-projecting pixels of the color picture described by the color data at a depth value determined according to pixels of the depth picture described by the geometry data; and
on condition that the geometry data is of the second geometry type, use de-projection data for de-projecting pixels of the color picture described by the color data at a depth value determined by the parametric function of the geometry data, the parametric initialized with the list of parameter values.
12. The device of claim 11 , wherein the color data further comprises a description of a size and/or of a shape of the color picture in the color image.
13. The device of claim 11 , wherein the identifier of the parametric function points to a function of a list of parametric functions stored in memory.
14. The device of claim 11 , wherein the stream comprises a sequence of color images, depth images and associated metadata, a depth image and related metadata being associated with a color image, the color images being structured by temporal information.
15. A device for encoding a three-dimensional scene in a data stream, the device comprising a memory associated with a processor configured to:
obtain a set of patches, each patch comprising:
de-projection data;
a color picture; and
a geometry data comprising:
a first geometry type comprising a depth picture; or
geometry data of a second geometry type comprising an identifier of a parametric function and a list of parameter values for the identified parametric function;
generate a color image by packing color pictures of the patches and a color data comprising a description of a location of the color picture as packed in the color image;
generate a depth image by packing depth pictures of patches having a geometry data of the first geometry type and replacing the depth picture in the geometry data of corresponding patch by a description of a location of the depth picture of the patch as packed in the depth image; and
encode in the stream:
the generated color image;
the generated depth image; and
metadata comprising the de-projection data, the color data, and the geometry data of every patch.
16. The device of claim 15 , wherein the color data further comprises a description of a size and/or of a shape of the color picture in the color image.
17. The device of claim 15 , wherein the identifier of the parametric function points to a function of a list of parametric functions stored in memory.
18. The device of claim 15 , wherein the stream comprises a sequence of color images, depth images and associated metadata, a depth image and related metadata being associated with a color image, the color images being structured by temporal information.