Method and device for rendering point cloud-based data

1. A method of rendering point cloud-based volumetric data, the method comprising:

receiving, at a computing device, a plurality of two-dimensional (2D) image frames corresponding to a three-dimensional (3D) point cloud, and projection metadata comprising data of projecting the 3D point cloud into a 2D image plane;

reconstructing a plurality of 3D points corresponding to the 3D point cloud, based on the plurality of 2D image frames and the projection metadata;

generating texture images based on the plurality of 3D points; and

rendering at least one quad image based on the texture images,

wherein each of the plurality of 2D image frames comprises a plurality of pixels, and

wherein the reconstructing the plurality of 3D points comprises:

identifying redundant pixels from each of the plurality of 2D image frames based on occupancy metadata corresponding to each of the plurality of 2D image frames, wherein the occupancy metadata comprises information of points located on a line lying in a direction of the projecting the 3D point cloud into the 2D image plane, information of the 3D points among the plurality of 3D points concentrated on a small area of which length is predetermined, information of a number of the points located on the line lying in the direction of the projecting the 3D point cloud into the 2D image plane, binary information about occupant pixels indicating pixels being valid occupied pixels, and binary information about the redundant pixels indicating pixels being redundant pixels; and

removing the redundant pixels from the plurality of 2D image frames.

2. The method of claim 1 , wherein the occupancy metadata further comprises information about a number of the redundant pixels.

3. The method of claim 1 , wherein the reconstructing the plurality of 3D points further comprises:

identifying non-redundant pixels among each of the plurality of 2D image frames by discarding the redundant pixels; and

determining locations of each of the non-redundant pixels in a 3D space based on the projection metadata.

4. The method of claim 1 , wherein the redundant pixels comprise a set of pixels excluding at least one point, and

wherein the set of pixels and the at least one point correspond to the points located on the line lying in the direction of the projecting the 3D point cloud into the 2D image plane and a number of the set of pixels.

5. The method of claim 1 , wherein the information of points located on the line lying in the direction of the projecting the 3D point cloud into the 2D image plane comprises coordinate information of the points located on the line lying in the direction of the projecting the 3D point cloud into the 2D image plane.

6. The method of claim 1 , wherein the projection metadata comprises geometry information of the plurality of 3D points, visual information of the plurality of 3D points, position information of pixels corresponding to the plurality of 3D points in a 3D space, and color information of the pixels corresponding to the plurality of 3D points.

7. The method of claim 1 , wherein the projection metadata comprises at least one of 3D coordinate values of 3D point cloud in a 3D space when projecting the 3D point cloud into the 2D image plane, 2D coordinate values of 3D point cloud in a 2D space when each of the 3D point cloud is projected onto the 2D image plane, or linking information which links the 3D coordinate values and the 2D coordinate values.

8. The method of claim 1 , further comprising:

generating, based on screen space projection data, screen space projection image by projecting the plurality of 3D points to a screen space domain, wherein the screen space projection data comprises at least one of information about one or more camera parameters or one or more perspective projection parameters.

9. The method of claim 8 , further comprising:

obtaining viewing information associated with a viewing application requesting rendering of the plurality of 3D points, wherein the viewing information comprises a position of a reference viewing point, an orientation of the reference viewing point, and information associated with a field of view determined from the reference viewing point;

obtaining screen space projection based on the viewing information associated with the viewing application;

discarding, from the plurality of 3D points, one or more 3D points that are projected outside the screen space projection to obtain a first subset of 3D points placed on at least one line of sight associated with the screen space projection;

performing a depth test on the first subset of 3D points to obtain a second subset of 3D points; and

projecting the second subset of 3D points to the screen space domain to generate the screen space projection image.

10. The method of claim 9 , wherein the projecting the second subset of 3D points to the screen space domain comprises:

projecting the second subset of 3D points to the screen space domain based on the screen space projection data.

11. The method of claim 10 , wherein the one or more camera parameters comprise an upward vector, a view direction, and a camera location, and the one or more perspective projection parameters comprise a field of view, a near plane location and a far plane location.

12. An apparatus for rendering point cloud-based volumetric data, the apparatus comprising:

a memory storing instructions;

at least one processor configured to execute the instructions to:

receive a plurality of two-dimensional (2D) image frames corresponding to a three-dimensional (3D) point cloud, and projection metadata comprising data of projecting the 3D point cloud into a 2D image plane,

reconstruct a plurality of 3D points corresponding to the 3D point cloud, based on the plurality of 2D image frames and the projection metadata, and

generate texture images based on the plurality of 3D points; and

a display configured to render at least one quad image based on the texture images,

wherein each of the plurality of 2D image frames comprises a plurality of pixels, and

wherein the at least one processor is further configured to execute the instructions to:

identify redundant pixels from each of the plurality of 2D image frames based on occupancy metadata corresponding to each of the plurality of 2D image frames, wherein the occupancy metadata comprises information of points located on a line lying in a direction of the projecting the 3D point cloud into the 2D image plane, information of the 3D points among the plurality of 3D points concentrated on a small area of which length is predetermined, information of a number of the points located on the line lying in the direction of the projecting the 3D point cloud into the 2D image plane, binary information about occupant pixels indicating pixels being valid occupied pixels, and binary information about the redundant pixels indicating pixels being redundant pixels; and

remove the redundant pixels from the plurality of 2D image frames.

13. The apparatus of claim 12 , wherein the at least one processor is further configured to execute the instructions to:

identify non-redundant pixels among each of the plurality of 2D image frames by discarding the redundant pixels, and

determine locations of each of the non-redundant pixels in a 3D space based on the projection metadata.

14. The apparatus of claim 12 , wherein the at least one processor is further configured to execute the instructions to generate, based on screen space projection data, screen space projection image by projecting the plurality of 3D points to a screen space domain, and

wherein the screen space projection data comprises at least one of information about one or more camera parameters or one or more perspective projection parameters.

15. The apparatus of claim 14 , wherein the at least one processor is further configured to execute the instructions to:

obtain viewing information associated with a viewing application requesting rendering of the plurality of 3D points, wherein the viewing information comprises a position of a reference viewing point, an orientation of the reference viewing point, and information associated with a field of view determined from the reference viewing point,

obtain screen space projection based on the viewing information associated with the viewing application,

discard, from the plurality of 3D points, one or more 3D points that are projected outside the screen space projection to obtain a first subset of 3D points placed on at least one line of sights associated with the screen space projection,

perform a depth test on the first subset of 3D points to obtain a second subset of 3D points, and

project the second subset of 3D points to the screen space domain to generate the screen space projection image.

16. The apparatus of claim 15 , wherein the at least one processor is further configured to execute the instructions to project the second subset of 3D points to the screen space domain based on the screen space projection data.

17. A non-transitory computer readable medium storing computer readable instructions that are executable by a processor to perform a method of rendering point cloud-based volumetric data, the method comprising:

receiving, at a computing device, a plurality of two-dimensional (2D) image frames corresponding to a three-dimensional (3D) point cloud, and projection metadata comprising data of projecting the 3D point cloud into a 2D image plane;

reconstructing a plurality of 3D points corresponding to the 3D point cloud, based on the plurality of 2D image frames and the projection metadata;

generating texture images based on the plurality of 3D points; and

rendering at least one quad image based on the texture images,

wherein each of the plurality of 2D image frames comprises a plurality of pixels, and

wherein the reconstructing the plurality of 3D points comprises:

identifying redundant pixels from each of the plurality of 2D image frames based on occupancy metadata corresponding to each of the plurality of 2D image frames, wherein the occupancy metadata comprises information of points located on a line lying in a direction of the projecting the 3D point cloud into the 2D image plane, information of the 3D points among the plurality of 3D points concentrated on a small area of which length is predetermined, information of a number of the points located on the line lying in the direction of the projecting the 3D point cloud into the 2D image plane, binary information about occupant pixels indicating pixels being valid occupied pixels, and binary information about the redundant pixels indicating pixels being redundant pixels; and

removing the redundant pixels from the plurality of 2D image frames.