Attribute transfer in V-PCC
A method for point cloud decoding includes receiving a bitstream. The method also includes decoding the bitstream into multiple frames that include pixels. Certain pixels of the multiple frames correspond to points of a three-dimensional (3D) point cloud. The multiple frames include a first set of frames that represent locations of the points of the 3D point cloud and a second set of frames that represent attribute information for the points of the 3D point cloud. The method further includes reconstructing the 3D point cloud based on the first set of frames. Additionally, the method includes identifying a first portion of the points of the reconstructed 3D point cloud based at least in part on a property associated with the multiple frames. The method also includes modifying a portion of the attribute information. The portion of the attribute information that is modified corresponds to the first portion of the points.
1. A device comprising:
a communication interface configured to receive a bitstream; and
a processor operably coupled to the communication interface, wherein the processor is configured to:
identify, based on the bitstream, geometry frames that represent positions of points of a three dimensional (3D) point cloud and attribute frames that represent attribute values of the points of the 3D point cloud;
reconstruct the 3D point cloud based on at least the geometry frames;
perform a geometry smoothing process on the reconstructed 3D point cloud;
determine whether a position of a reconstructed point in the reconstructed 3D point cloud was modified as part of the geometry smoothing process based on an array associated with the reconstructed point;
determine whether to update an attribute value of the reconstructed point based on whether the position of the reconstructed point was modified as part of the geometry smoothing process; and
perform an attribute transfer process to update the attribute value of the reconstructed point based on a determination that the position of the reconstructed point was modified as part of the geometry smoothing process.
2. The device of claim 1 , wherein:
the array is an output of the geometry smoothing process denoted as isModifiedPoint[n] or isUpdatedPoint[n],
a value of 0 for the array denotes that a position of an n-th reconstructed point was not modified as part of the geometry smoothing process, and
a value of 1 for the array denotes that a position of the n-th reconstructed point was modified as part of the geometry smoothing process.
3. The device of claim 1 , wherein the processor is further configured to:
determine whether a second position of a second reconstructed point in the reconstructed 3D point cloud was modified as part of the geometry smoothing process based on a second array associated with the second reconstructed point;
determine whether to update a second attribute value of the second reconstructed point based on whether the second position of the second reconstructed point was modified as part of the geometry smoothing process; and
retain the second attribute value of the second reconstructed point based on a determination that the second position of the second reconstructed point was not modified as part of the geometry smoothing process.
4. The device of claim 1 , wherein the processor is further configured to:
identify a value for the array as initialized to 0;
determine that the position of the reconstructed point in the reconstructed 3D point cloud was modified as part of the geometry smoothing process; and
set the value for the array to 1 to indicate that the position of the reconstructed point in the reconstructed 3D point cloud was modified.
5. The device of claim 1 , wherein the processor is further configured to determine whether the position of the reconstructed point in the reconstructed 3D point cloud was modified based on a comparison of a threshold against a variable indicative of a difference between a position of a point in the smoothed 3D point cloud and a position of a corresponding point of the reconstructed 3D point cloud.
6. The device of claim 1 , wherein the processor is further configured to determine to perform the smoothing process on the reconstructed 3D point cloud in response to an identification that the bitstream includes an indication that geometry smoothing is enabled.
7. The device of claim 6 , wherein the indication that geometry smoothing is enabled comprises a geometry smoothing SEI message having a syntax element set to a first value.
8. The device of claim 1 , wherein the processor is configured to:
identify a set of neighboring points of the reconstructed point; and
determine the modified attribute value for the reconstructed point based on a weighted average of attribute values of the identified set of neighboring points.
9. The device of claim 1 , wherein the processor is configured to
perform a forward search to identify forward neighboring points of the reconstructed point; and
determine the modified attribute value for the reconstructed point based on a weighted average of attribute values of the identified forward neighboring points.
10. The device of claim 1 , wherein the processor is configured to:
perform a backward search to identify backward neighboring points of the reconstructed point; and
determine the modified attribute value for the reconstructed point based on a weighted average of attribute values of the identified backward neighboring points.
11. A method comprising:
receiving a bitstream;
identifying, based on the bitstream, geometry frames that represent positions of points of a three dimensional (3D) point cloud and attribute frames that represent attribute values of the points of the 3D point cloud;
reconstructing the 3D point cloud based on at least the geometry frames;
performing a geometry smoothing process on the reconstructed 3D point cloud;
determining whether a position of a reconstructed point in the reconstructed 3D point cloud was modified as part of the geometry smoothing process based on an array associated with the reconstructed point;
determining whether to update an attribute value of the reconstructed point based on whether the position of the reconstructed point was modified as part of the geometry smoothing process; and
performing an attribute transfer process to update the attribute value of the reconstructed point based on a determination that the position of the reconstructed point was modified as part of the geometry smoothing process.
12. The method of claim 11 , wherein:
the array is an output of the geometry smoothing process denoted as isModifiedPoint[n] or isUpdatedPoint[n],
a value of 0 for the array denotes that a position of an n-th reconstructed point was not modified as part of the geometry smoothing process, and
a value of 1 for the array denotes that a position of the n-th reconstructed point was modified as part of the geometry smoothing process.
13. The method of claim 11 , further comprising:
determining whether a second position of a second reconstructed point in the reconstructed 3D point cloud was modified as part of the geometry smoothing process based on a second array associated with the second reconstructed point;
determining whether to update a second attribute value of the second reconstructed point based on whether the second position of the second reconstructed point was modified as part of the geometry smoothing process; and
retaining the second attribute value of the second reconstructed point based on a determination that the second position of the second reconstructed point was not modified as part of the geometry smoothing process.
14. The method of claim 11 , further comprising:
identifying a value for the array as initialized to 0;
determining that the position of the reconstructed point in the reconstructed 3D point cloud was modified as part of the geometry smoothing process; and
setting the value for the array to 1 to indicate that the position of the reconstructed point in the reconstructed 3D point cloud was modified.
15. The method of claim 11 , further comprising:
determining whether the position of the reconstructed point in the reconstructed 3D point cloud was modified based on a comparison of a threshold against a variable indicative of a difference between a position of a point in the smoothed 3D point cloud and a position of a corresponding point of the reconstructed 3D point cloud.
16. The method of claim 11 , further comprising:
determining to perform the smoothing process on the reconstructed 3D point cloud in response to an identification that the bitstream includes an indication that geometry smoothing is enabled.
17. The method of claim 16 , wherein the indication that geometry smoothing is enabled comprises a geometry smoothing SEI message having a syntax element set to a first value.
18. The method of claim 11 , further comprising:
identifying a set of neighboring points of the reconstructed point; and
determining the modified attribute value for the reconstructed point based on a weighted average of attribute values of the identified set of neighboring points.
19. The method of claim 11 , further comprising:
performing a forward search to identify forward neighboring points of the reconstructed point; and
determining the modified attribute value for the reconstructed point based on a weighted average of attribute values of the identified forward neighboring points.
20. The method of claim 11 , further comprising:
performing a backward search to identify backward neighboring points of the reconstructed point; and
determining the modified attribute value for the reconstructed point based on a weighted average of attribute values of the identified backward neighboring points.