Method and apparatus for geometry smoothing by local geometry projection

1. A method of decoding a video stream encoded using video point cloud coding, the method being performed by at least one processor and comprising:

obtaining a geometry-reconstructed point cloud;

dividing the geometry-reconstructed point cloud into a plurality of cells, wherein a cell of the plurality of cells comprises a first plurality of points;

selecting a plane representing the first plurality of points based on an average position p of the first plurality of points;

generating a second plurality of points by projecting the first plurality of points onto the plane;

obtaining a smoothed geometry-reconstructed point cloud based on the second plurality of points; and

reconstructing a dynamic point cloud using the smoothed geometry-reconstructed point cloud,

wherein the plane is selected based on a determination that the plane passes through p and has a same direction as an eigenvector of a covariance matrix of the first plurality of points.

2. The method of claim 1 , wherein p is determined as follows:

p

_

=

Σ

i

=

1

N

⁢

x

i

N

wherein x i represents a point of the first plurality of points having an index i, and N represents a number of the first plurality of points.

3. The method of claim 2 , wherein the covariance matrix of the first plurality of points is determined as follows:

C

=

Σ

i

=

1

N

⁡

(

x

i

-

p

_

)

⁢

(

x

i

-

p

_

)

T

N

wherein C represents the covariance matrix,

wherein T represents a transverse operation,

wherein eigenvalues of C are represented as λ 3 <λ 2 <λ 1

wherein eigenvectors of λ 3 , λ 2 , and λ 1 are represented as v 3 , v 2 , v 1 , and

wherein the plane is selected based on p and v3.

4. The method of claim 3 , wherein the plane passes through p and has a same direction as v 3 .

5. The method of claim 4 , wherein a point of the second plurality of points is determined as follows:

X proj =[( X− p )· v 1 ] v 1 +[( X− p )· v 2 ] v 2 + p

wherein X represents a point of the first plurality of points, and X proj represents the point of the second plurality of points.

6. The method of claim 5 , wherein the point of the first plurality of points is moved to a weighted projection as follows:

X→αX proj +(1−α) X

wherein α represents a projection weight.

7. The method of claim 1 , wherein the plane comprises a flat manifold representing the first plurality of points.

8. The method of claim 1 , wherein metadata of the video stream indicates at least one from among an indication of whether smoothing is used, an indication of a size of the cell, and a projection weight used in the projection.

9. An apparatus for decoding a video stream encoded using video point cloud coding, the apparatus comprising:

at least one memory configured to store program code; and

at least one processor configured to read the program code and operate as instructed by the program code, the program code comprising:

first obtaining code configured to cause the at least one processor to obtain a geometry-reconstructed point cloud;

dividing code configured to cause the at least one processor to divide the geometry-reconstructed point cloud into a plurality of cells, wherein a cell of the plurality of cells comprises a first plurality of points;

selecting code configured to cause the at least one processor to select a plane representing the first plurality of points based on an average position p of the first plurality of points;

generating code configured to cause the at least one processor to generate a second plurality of points by projecting the first plurality of points onto the plane;

second obtaining code configured to cause the at least one processor to obtain a smoothed geometry-reconstructed point cloud based on the second plurality of points; and

reconstructing code configured to cause the at least one processor to reconstruct a dynamic point cloud using the smoothed geometry-reconstructed point cloud,

wherein the plane is selected based on a determination that the plane passes through p and has a same direction as an eigenvector of a covariance matrix of the first plurality of points.

10. The apparatus of claim 9 , wherein p is determined as follows:

p

_

=

Σ

i

=

1

N

⁢

x

i

N

wherein x i represents a point of the first plurality of points having an index i, and N represents a number of the first plurality of points.

11. The apparatus of claim 10 , wherein the covariance matrix of the first plurality of points is determined as follows:

C

=

Σ

i

=

1

N

⁡

(

x

i

-

p

_

)

⁢

(

x

i

-

p

_

)

T

N

wherein C represents the covariance matrix,

wherein T represents a transverse operation,

wherein eigenvalues of C are represented as λ 3 <λ 2 <λ 1

wherein eigenvectors of λ 3 , λ 2 , and λ 1 are represented as v 3 , v 2 , v 1 , and

wherein the plane is selected based on p and v3.

12. The apparatus of claim 11 , wherein the plane passes through p and has a same direction as v 3 .

13. The apparatus of claim 12 , wherein a point of the second plurality of points is determined as follows:

X proj =[( X− p )· v 1 ] v 1 +[( X− p )· v 2 ] v 2 + p

wherein X represents a point of the first plurality of points, and X proj represents the point of the second plurality of points.

14. The apparatus of claim 13 , wherein the point of the first plurality of points is moved to a weighted projection as follows:

X→αX proj +(1−α) X

wherein α represents a projection weight.

15. The apparatus of claim 9 , wherein the plane comprises a flat manifold representing the first plurality of points.

16. The apparatus of claim 9 , wherein metadata of the video stream indicates at least one from among an indication of whether smoothing is used, an indication of a size of the cell, and a projection weight used in the projection.

17. A non-transitory computer-readable medium storing computer instructions decoding a video stream encoded using video point cloud coding that, when executed by at least one processor, cause the at least one processor to:

obtain a geometry-reconstructed point cloud;

divide the geometry-reconstructed point cloud into a plurality of cells, wherein a cell of the plurality of cells comprises a first plurality of points;

select a plane representing the first plurality of points based on an average position p of the first plurality of points;

generate a second plurality of points by projecting the first plurality of points onto the plane;

obtain a smoothed geometry-reconstructed point cloud based on the second plurality of points; and

reconstruct a dynamic point cloud using the smoothed geometry-reconstructed point cloud,

wherein the plane is selected based on a determination that the plane passes through p and has a same direction as an eigenvector of a covariance matrix of the first plurality of points.

18. The non-transitory computer-readable medium of claim 17 , wherein p is determined as follows:

p

_

=

Σ

i

=

1

N

⁢

x

i

N

wherein x i represents a point of the first plurality of points having an index i, and N represents a number of the first plurality of points.

19. The non-transitory computer-readable medium of claim 18 , wherein the covariance matrix of the first plurality of points is determined as follows:

C

=

Σ

i

=

1

N

⁡

(

x

i

-

p

_

)

⁢

(

x

i

-

p

_

)

T

N

wherein C represents the covariance matrix,

wherein T represents a transverse operation,

wherein eigenvalues of C are represented as λ 3 <λ 2 <λ 1

wherein eigenvectors of λ 3 , λ 2 , and λ 1 are represented as v 3 , v 2 , v 1 , and

wherein the plane is selected based on p and v3.

20. The non-transitory computer-readable medium of claim 19 , wherein the plane passes through p and has a same direction as v 3 .