Methods, devices and stream for encoding and decoding volumetric video
A sequence of point clouds is encoded as a video by an encoder and transmitted to a decoder which retrieves the sequence of point clouds. Visible points of a point cloud are iteratively projected on projection maps according to at least two centers of projection, to determine a patch data item lists. One of the centers of projection is selected and corresponding image patches are generated and packed into a picture. Pictures and associated patch data item list are encoded in a stream. The decoding method decodes pictures and associated patch data item lists. Pixels of image patches comprised in pictures are un-projected according to data stored in associated patches. The methods have the advantage of encoding every point of point clouds in a manner avoiding artifacts and allowing decoding at video frame rate.
1. A method of encoding a point cloud into a stream, the method comprising:
obtaining a group of centers of projection determined according to a viewing point relative to the point cloud;
determining, for said point cloud, a list of patch data items by projecting on a projection map, points of said point cloud visible from one center of projection of said group of centers of projection and clustering adjacent pixels as image patches, a patch data item being associated with an image patch and comprising:
information identifying said one center of projection;
a description of a region of the space, said region being defined according to said one center of projection, an angular range and a depth range; and,
information identifying an area for the associated image patch within a picture;
generating said picture comprising a set of image patches; and
encoding, in said stream, the picture and the list of patch data items.
2. The method of claim 1 , wherein determining, for said point cloud, a list of data items comprises:
for a first center of projection and a different second centers of projection among the group of centers of projection:
determining a first list of patch data items by projecting on a first projection map, points of the point cloud visible from the first center of projection and by clustering adjacent pixels of the first projection map according to depth information;
determining a second list of patch data items by projecting on a second projection map, points of the point cloud visible from the second center of projection and by clustering adjacent pixels of the second projection map according to depth information;
selecting one of the first and second list of patch data items and adding the selected list of patch data items to the list of patch data items; and
removing the points of the point cloud visible from the center of projection of the selected list; and
iterating the above steps until the point cloud is empty or the patch data item list is full.
3. The method of claim 2 , wherein said point cloud comprises points of a group of point clouds of a sequence of point clouds, a list of patch data items being determined for a group of pictures, each picture of said group of pictures being generated for a point cloud of said group of point clouds, and said encoding comprising encoding, in said stream, the group of pictures and the list of patch data items.
4. The method of claim 2 , wherein a pixel of an image patch comprises a depth value, said depth value being determined according to depth range of the patch data item associated with said image patch.
5. The method of claim 4 , wherein said a pixel of the image patches further comprises a color value.
6. The method of claim 5 , wherein depth values are stored by pixels of a first picture and color values are stored by pixels of a second picture, first and second pictures being paired and encoded into the stream.
7. A device for encoding a point cloud into a stream, the device comprising a memory associated with at least one processor configured to:
obtain a group of centers of projection determined according to a viewing point;
determine, for said point cloud, a list of patch data items by projecting on a projection map, points of said point cloud visible from one center of projection of said group of centers of projection and clustering adjacent pixels as image patches, a patch data item being associated with an image patch and comprising:
information identifying said one center of projection;
a description of a region of the space, said region being defined according to said one center of projection, an angular range and a depth range; and,
information identifying an area for the associated image patch within a picture;
generate said picture comprising a set of image patches; and
encode, in said stream, the picture and the list of patch data items.
8. The device of claim 7 , wherein said at least one processor is configured to determine, for said point cloud, a list of data items by:
for a first and a different second center of projection among the group of centers of projection:
determining a first list of patch data items by projecting on a first projection map, points of the point cloud visible from the first center of projection and by clustering adjacent pixels of the first projection map according to depth information;
determining a second list of patch data items by projecting on a second projection map, points of the point cloud visible from the second center of projection and by clustering adjacent pixels of the second projection map according to depth information;
selecting one of the first and second list of patch data items and adding the selected list of patch data items to the list of patch data items; and
removing the points of the point cloud visible from the center of projection of the selected list; and
iterating the above steps until the point cloud is empty or the patch data item list is full.
9. The device of claim 8 , wherein said point cloud comprises points of a group of point clouds of a sequence of point clouds, a list of patch data item list being determined for a group of pictures, each picture of said group of pictures being generated for a point cloud of said group of point clouds, said at least one processor being configured to encode, in said stream, the group of pictures and the list of patch data items.
10. The device of claim 8 , wherein a pixel of an image patch comprises a depth value, said depth value being determined according to depth range of the patch data item associated with said image patch.
11. A method of decoding a point cloud from a stream, the method comprising:
obtaining a group of centers of projection determined according to a viewing point relative to the point cloud;
decoding a picture packing image patches and a list of patch data items from the stream, a patch data item comprising:
information identifying one center of projection within the group of centers of projection;
a description of a region of the space, said region being defined according to the one center of projection, an angular range and a depth range; and,
information identifying an area of an image patch within the picture;
decoding points of said point cloud by un-projecting pixels of each image patch according to the associated patch data item.
12. The method of claim 11 , wherein coordinates of the centers of projection of the group of centers of projection are obtained from a memory.
13. A device for decoding a point cloud from a stream, the device comprising a memory associated with at least a processor configured to:
obtain a group of centers of projection determined according to a viewing point relative to the point cloud;
decode a picture packing image patches and a list of patch data items from the stream, a patch data item comprising:
information identifying one center of projection within the group of centers of projection;
a description of a region of the space, said region being defined according to the one center of projection, an angular range and a depth range; and,
information identifying an area of an image patch within the picture;
decode points of said point cloud by un-projecting pixels of each image patch according to the associated patch data item.
14. The device of claim 13 , wherein coordinates of the centers of projection of the group of centers of projection are obtained from the stream.