State summarization for binary voxel grid coding
In one implementation, we improve the binary voxel-based octree coding method, via a proposed state summarization module for context modeling. Given a current voxel to be encoded or decoded, instead of directly estimating its occupancy probability based on the associated binary occupancy context, a proposed state summarization module is applied to convert the original binary context to a summarized representation. Under the summarized representation, the estimation of the occupancy probability becomes more affordable and effective. In particular, density-based state summarization, pattern-based, learning-based state summarization, and learning-based state summarization methods are provided.
1 . An apparatus for encoding point cloud data, comprising:
at least one processor configured for
determining a first set of a first number of states associated with an occupancy state for each of a plurality of encoded voxels neighboring a current voxel, wherein the current voxel and the plurality of neighboring encoded voxels are in a point cloud represented by the point cloud data;
determining a density of occupied voxels in the plurality of neighboring encoded voxels;
processing the first set of states to obtain a second set of a second number of states, based on the density of occupied voxels in the plurality of neighboring encoded voxels;
predicting, based on the second set of states, a probability for an occupancy state for the current voxel; and
encoding the occupancy state for the current voxel, based on said predicted probability for the occupancy state for the current voxel.
2 . The apparatus of claim 1 , wherein processing the first set of states comprises converting the first set of states to a summarized state space having the second number of states.
3 . The apparatus of claim 1 , wherein determining the density of occupied voxels comprises:
classifying the voxels neighboring the current voxel into one or more classes of neighboring voxels based on a distance from each neighboring voxel to the current voxel; and
determining a number of occupied neighboring voxels included in each of the one or more classes of neighboring voxels.
4 . The apparatus of claim 2 , wherein converting the first set of states to the summarized state space comprises determining that one or more of a set of patterns exists in a voxel neighborhood comprising the plurality of neighboring voxels.
5 . The apparatus of claim 1 , wherein processing the first set of states comprises applying a neural network including a plurality of point-based MLP layers.
6 . The apparatus of claim 1 , wherein processing the first set of states comprises applying a first neural network including a plurality of convolutional layers and a second neural network including a plurality of point-based MLP layers, wherein outputs from said first and second neural networks are concatenated.
7 . A method for encoding point cloud data, comprising:
determining a first set of a first number of states associated with an occupancy state for each of a plurality of encoded voxels neighboring a current voxel, wherein the current voxel and the plurality of encoded neighboring voxels are in a point cloud represented by point cloud data;
determining a density of occupied voxels in the plurality of neighboring encoded voxels;
processing the first set of states to obtain a second set of a second number of states, based on the density of occupied voxels in the plurality of neighboring encoded voxels;
predicting, based on the second set of states, a probability for an occupancy state for the current voxel; and
encoding the occupancy state for the current voxel, based on said predicted probability for the occupancy state for the current voxel.
8 . The method of claim 7 , wherein determining the density of occupied voxels comprises:
classifying the voxels neighboring the current voxel into one or more classes of neighboring voxels based on a distance from each neighboring voxel to the current voxel; and
determining a number of occupied neighboring voxels included in each of the one or more classes of neighboring voxels.
9 . The method of claim 7 , wherein processing the first set of states comprises converting the first set of states to a summarized state space having the second number of states.
10 . The method of claim 9 , wherein converting the first set of states to the summarized state space comprises determining that one or more of a set of patterns exists in a voxel neighborhood comprising the plurality of neighboring voxels.
11 . The method of claim 7 , wherein processing the first set of states comprises applying a neural network including a plurality of point-based MLP layers.
12 . The method of claim 7 , wherein processing the first set of states comprises applying a first neural network including a plurality of convolutional layers and a second neural network including a plurality of point-based MLP layers, wherein outputs from said first and second neural networks are concatenated.
13 . An apparatus for decoding point cloud data, comprising:
at least one processor configured for
determining a first set of a first number of states associated with an occupancy state for each of a plurality of decoded voxels neighboring a current voxel, wherein the current voxel and the plurality of neighboring decoded voxels are in a point cloud represented by the point cloud data;
determining a density of occupied voxels in the plurality of neighboring encoded voxels;
processing the first set of states to obtain a second set of a second number of states, based on the density of occupied voxels in the plurality of neighboring encoded voxels;
predicting, based on the second set of states, a probability for an occupancy state for the current voxel; and
decoding the occupancy state for the current voxel, based on said predicted probability for the occupancy state for the current voxel.
14 . The apparatus of claim 13 , wherein processing the first set of states comprises converting the first set of states to a summarized state space having the second number of states.
15 . The apparatus of claim 14 , wherein converting the first set of states to the summarized state space comprises determining that one or more of a set of patterns exists in a voxel neighborhood comprising the plurality of neighboring voxels.
16 . The apparatus of claim 13 , wherein processing the first set of states comprises applying a neural network including a plurality of point-based MLP layers.
17 . A method for decoding point cloud data, comprising:
determining a first set of a first number of states associated with an occupancy state for each of a plurality of decoded voxels neighboring a current voxel, wherein the current voxel and the plurality of decoded neighboring voxels are in a point cloud represented by the point cloud data;
determining a density of occupied voxels in the plurality of neighboring encoded voxels;
processing the first set of states to obtain a second set of a second number of states, based on the density of occupied voxels in the plurality of neighboring encoded voxels;
predicting, based on the second set of states, a probability for an occupancy state for the current voxel; and
decoding the occupancy state for the current voxel, based on said predicted probability for the occupancy state for the current voxel.
18 . The method of claim 17 , wherein processing the first set of states comprises converting the first set of states to a summarized state space having the second number of states.
19 . The method of claim 18 , wherein converting the first set of states to the summarized state space comprises determining that one or more of a set of patterns exists in a voxel neighborhood comprising the plurality of neighboring voxels.
20 . The method of claim 17 , wherein processing the first set of states comprises applying a neural network including a plurality of point-based MLP layers.