Predictive coding of point clouds using multiple frames of references
Methods and devices for encoding a point cloud. More than one frame of reference is identified and a transform defines the relative motion of a second frame of reference to a first frame of reference. The space is segmented into regions and each region is associated with one of the frames of reference. Local motion vectors within a region are expressed relative to the frame of reference associated with that region. Occupancy of the bitstream is entropy encoded based on predictions determined using the location motion vectors and the transform associated with the attached frame of reference.
1. A method of encoding a point cloud, the point cloud being located within a volumetric space containing the points of the point cloud, each of the points having a geometric location within the volumetric space, the method comprising:
determining a transform defining relative motion of a second frame of reference to a first frame of reference;
segmenting the volumetric space into regions, each region being associated with one of the frames of reference;
for a cuboid in one of the regions, generating a predicted sub-volume based on previously-encoded point cloud data and a local motion vector expressed relative to the frame of reference associated with said one of the regions, wherein generating the predicted sub-volume includes applying the transform to the previously-encoded point cloud data to generate transformed previously-encoded point cloud data, and applying the local motion vector to the transformed previously-encoded point cloud data to generate the predicted sub-volume;
entropy encoding occupancy of the cuboid based in part on the predicted sub-volume; and
outputting a bitstream of encoded data including the entropy encoded occupancy of the cuboid, the local motion vector, and the transform.
2. The method claimed in claim 1 , wherein the first frame of reference is fixed to a vehicle and wherein the second frame of reference is fixed to Earth.
3. The method claimed in claim 1 , wherein segmenting includes segmenting at least a portion of the volumetric space into prediction units and assigning each prediction unit to one of the regions.
4. The method claimed in claim 1 , wherein the transform includes a 3D matrix and a 3D vector.
5. The method claimed in claim 1 , further comprising determining a list of frames of reference and encoding the list, and wherein segmenting includes, for each region, encoding an index to the list that associates that region with said one of the frame of reference from the list.
6. An encoder for encoding a point cloud to generate a bitstream of compressed point cloud data, the point cloud being located within a volumetric space recursively split into sub-volumes and containing the points of the point cloud, each of the points having a geometric location within the volumetric space, the encoder comprising:
a processor;
memory; and
an encoding application containing instructions executable by the processor that, when executed, cause the processor to:
determine a transform defining relative motion of a second frame of reference to a first frame of reference;
segment the volumetric space into regions, each region being associated with one of the frames of reference;
for a cuboid in one of the regions, generate a predicted sub-volume based on previously-encoded point cloud data and a local motion vector expressed relative to the frame of reference associated with said one of the regions wherein to generate the predicted sub-volume by applying the transform to the previously-encoded point cloud data to generate transformed previously-encoded point cloud data, and applying the local motion vector to the transformed previously-encoded point cloud data to generate the predicted sub-volume;
entropy encode occupancy of the cuboid based in part on the predicted sub-volume; and
output a bitstream of encoded data including the entropy encoded occupancy of the cuboid, the local motion vector, and the transform.
7. A decoder for reconstructing a point cloud from a bitstream of compressed point cloud data, the point cloud being located within a volumetric space recursively split into sub-volumes and containing the points of the point cloud, each of the points having a geometric location within the volumetric space, the decoder comprising:
a processor;
memory; and
a decoding application containing instructions executable by the processor that, when executed, cause the processor to
decode a bitstream to reconstruct a transform defining relative motion of a second frame of reference to a first frame of reference;
segment the volumetric space into regions, each region being associated with one of the frames of reference;
for a cuboid in one of the regions, decode the bitstream to obtain a local motion vector expressed relative to the frame of reference associated with said one of the regions, and generating a predicted sub-volume based on previously-encoded point cloud data and the decoded local motion vector, wherein to generate the predicted sub-volume by applying the transform to the previously-encoded point cloud data to generate transformed previously-encoded point cloud data, and applying the local motion vector to the transformed previously-encoded point cloud data to generate the predicted sub-volume;
entropy decode the bitstream to reconstruct occupancy of the cuboid based in part on the predicted sub-volume; and
output the reconstructed point cloud based on the reconstructed occupancy of the cuboid.
8. A method of decoding encoded data to reconstruct a point cloud, the point cloud being located within a volumetric space containing the points of the point cloud, each of the points having a geometric location within the volumetric space, the method comprising:
decoding a bitstream to reconstruct a transform defining relative motion of a second frame of reference to a first frame of reference;
segmenting the volumetric space into regions, each region being associated with one of the frames of reference;
for a cuboid in one of the regions, decoding the bitstream to obtain a local motion vector expressed relative to the frame of reference associated with said one of the regions, and generating a predicted sub-volume based on previously-encoded point cloud data and the decoded local motion vector, wherein generating the predicted sub-volume includes applying the transform to the previously-encoded point cloud data to generate transformed previously-encoded point cloud data, and applying the local motion vector to the transformed previously-encoded point cloud data to generate the predicted sub-volume;
entropy decoding the bitstream to reconstruct occupancy of the cuboid based in part on the predicted sub-volume; and
outputting the reconstructed point cloud based on the reconstructed occupancy of the cuboid.
9. The method claimed in claim 8 , wherein the first frame of reference is fixed to a vehicle and wherein the second frame of reference is fixed to Earth.
10. The method claimed in claim 8 , wherein segmenting includes segmenting at least a portion of the volumetric space into prediction units and assigning each prediction unit to one of the regions.
11. The method claimed in claim 8 , wherein the transform includes a 3D matrix and a 3D vector.
12. The method claimed in claim 8 , further comprising decoding a list of frames of reference, and wherein segmenting includes, for each region, decoding an index to the list that associates that region with said one of the frame of reference from the list.