Method and apparatus for encoding and decoding volumetric content in and from a data stream
Methods and apparatus for encoding and decoding a volumetric scene are disclosed. A set of attribute and geometry patches is obtained by projecting samples of the volumetric scene onto the patches according to projection parameters. If the geometry patch is comparable to a planar layer located at a constant depth according to the projection parameters, only the attribute patch is packed in an attribute atlas image and the depth value is encoded in metadata. Otherwise, both attribute and geometry patches are packed in an atlas. At the decoding, if metadata for an attribute patch indicates that its geometry may be determined from the projection parameters and a constant depth, the attributes are inverse projected on a planar layer. Otherwise, attributes are inverse projected according to the associated geometry patch.
1. A method of synthetizing a viewport image, the method comprising:
obtaining, from a data stream, an attribute atlas image, a geometry atlas image, an atlas image packing patch pictures, a patch picture being a projection of a sample of a three-dimensional scene, and metadata comprising, for an attribute patch picture of the attribute atlas image:
projection parameters associated with the attribute patch picture, and
information indicating if the attribute patch picture is associated with a geometry patch picture of the geometry atlas image or if the attribute patch picture is associated with a depth value encoded in the metadata;
on a condition that an attribute patch picture is associated with a geometry patch picture, synthetizing the viewport image by inverse-projecting pixels of the attribute patch picture at a location determined by the geometry patch picture and projection parameters associated with the attribute patch picture; and
on a condition that an attribute patch picture is associated with a depth value, synthetizing the viewport image by inverse-projecting pixels of the attribute patch picture at a location determined by the depth value and projection parameters associated with the attribute patch picture.
2. The method of claim 1 , wherein pixels of the attribute atlas image encode two values for two different attributes, the two different attributes being inverse-projected together.
3. The method of claim 1 , wherein the data stream comprises two attribute atlases encoded according to a same packing layout, the metadata being generated for a pair of attribute patch pictures of each attribute atlases, the pair of attribute patch pictures being inverse-projected together.
4. A device for synthetizing the viewport image, the device comprising a processor configured for:
obtaining, from a data stream, an attribute atlas image, a geometry atlas image, an atlas image packing patch pictures, a patch picture being a projection of a sample of a three-dimensional scene, and metadata comprising, for an attribute patch picture of the attribute atlas image:
projection parameters associated with the attribute patch picture, and
information indicating if the attribute patch picture is associated with a geometry patch of the geometry atlas image or if the attribute patch picture is associated with a depth value encoded in the metadata;
on a condition that an attribute patch picture is associated with a geometry patch picture, synthetizing the viewport image by inverse-projecting pixels of the attribute patch picture at a location determined by the geometry patch picture and projection parameters associated with the attribute patch picture; and
on a condition that an attribute patch picture is associated with a depth value, synthetizing the viewport image by inverse-projecting pixels of the attribute patch picture at a location determined by the depth value and projection parameters associated with the attribute patch picture.
5. The device of claim 4 , wherein pixels of the attribute atlas image encode two values for two different attributes, the two different attributes being inverse-projected together.
6. The device of claim 4 , wherein the data stream comprises two attribute atlases encoded according to a same packing layout, the metadata being generated for a pair of attribute patch pictures of each attribute atlases, the pair of attribute patch pictures being inverse-projected together.
7. A method comprising:
obtaining a set of attribute patch pictures associated with a geometry patch picture, attribute and geometry patch pictures being obtained by projecting samples of a three-dimensional scene according to projection parameters;
for an attribute patch picture of the set of attribute patch pictures,
packing the attribute patch picture in an attribute atlas image; and
if the geometry patch picture associated with the attribute patch picture is comparable to a planar layer at a location determined by a depth value and the projection parameters, generating metadata comprising the projection parameters, the depth value and an information indicating that the attribute patch picture is associated with the depth value or,
in the other case, packing the geometry patch picture in a geometry atlas image and generating metadata comprising the projection parameters and information indicating that the attribute patch picture is associated with the geometry patch picture; and
encoding the attribute atlas image, the geometry atlas image and the metadata in a data stream.
8. The method of claim 7 , wherein pixels of the attribute atlas image encode two values for two different attributes.
9. The method of claim 7 , wherein the set of attribute patch pictures comprises pairs of attribute patch pictures for two different attributes, the pairs of attribute patch pictures being packed in two attribute atlases according to a same packing layout and the metadata being generated for a pair of attribute patch pictures.
10. A device comprising a processor configured for:
obtaining a set of attribute patch pictures associated with a geometry patch picture attribute and geometry patch pictures being obtained by projecting samples of a three-dimensional scene according to projection parameters;
for an attribute patch picture of the set of attribute patch pictures,
packing the attribute patch picture in an attribute atlas image; and
if the geometry patch picture associated with the attribute patch picture is comparable to a planar layer at a location determined by a depth value and the projection parameters, generating metadata comprising the projection parameters, the depth value and an information indicating that the attribute patch picture is associated with the depth value or,
in the other case, packing the geometry patch picture in a geometry atlas image and generating metadata comprising the projection parameters and information indicating that the attribute patch picture is associated with the geometry patch picture; and
encoding the attribute atlas image, the geometry atlas image and the metadata in a data stream.
11. The device of claim 10 , wherein pixels of the attribute atlas image encode two values for two different attributes.
12. The device of claim 10 , wherein the set of attribute patch pictures comprises pairs of attribute patch pictures for two different attributes, the pairs of attribute patch pictures being packed in two attribute atlases according to a same packing layout and the metadata being generated for a pair of attribute patch pictures.
13. A non-transitory computer readable medium containing an attribute atlas image, a geometry atlas image, an atlas image packing patch pictures, a patch picture being a projection of a sample of a three-dimensional scene, and metadata comprising, for an attribute patch picture of the attribute atlas image:
projection parameters associated with the attribute patch picture; and
information representative of an inverse-projecting mode indicating if the attribute patch picture is associated with a geometry patch of the geometry atlas image or if the attribute patch picture is associated with a depth value encoded in the metadata.
14. The non-transitory computer readable medium of claim 13 , wherein pixels of the attribute atlas image encode two values for two different attributes.
15. The non-transitory computer readable medium of claim 13 , comprising two attribute atlases encoded according to a same packing layout.
16. A non-transitory computer readable medium storing instructions that, when executed by one or more processors, perform the method of claim 1 .
17. The non-transitory computer readable medium of claim 16 , wherein pixels of the attribute atlas image encode two values for two different attributes, the two different attributes being inverse-projected together.
18. The non-transitory computer readable medium of claim 16 , wherein the data stream comprises two attribute atlases encoded according to a same packing layout, the metadata being generated for a pair of attribute patch pictures of each attribute atlases, the pair of attribute patch pictures being inverse-projected together.