Method of generating volume hologram using point cloud and mesh
View Patent ↗Disclosed is a method of generating a volume hologram using a point cloud and a mesh, in which a weight is given to a brightness of a light source according to a direction of a light in order to record a hologram of better quality. The method includes: (a) acquiring multi-view depth and color images; (b) generating point cloud data of a three-dimensional object from the acquired multi-view depth and color images; (c) generating mesh data of the three-dimensional object from the point cloud data of the three-dimensional object; (d) calculating a normal vector of each mesh from the mesh data of the three-dimensional object; (e) extracting three-dimensional data at a user viewpoint from the mesh data of the three-dimensional object by using the normal vector of the mesh; and (f) generating hologram data from three-dimensional data at the user viewpoint.
1. A method of generating a volume hologram using a point cloud and a mesh, the method comprising:
(a) acquiring multi-view depth and color images;
(b) generating point cloud data of a three-dimensional object from the acquired multi-view depth and color images;
(c) generating mesh data of the three-dimensional object from the point cloud data of the three-dimensional object;
(d) calculating a normal vector of each mesh from the mesh data of the three-dimensional object;
(e) extracting three-dimensional data at a user viewpoint from the mesh data of the three-dimensional object by using the normal vector of the mesh; and
(f) generating hologram data from the three-dimensional data at the user viewpoint,
wherein, in the step (f), the hologram data is generated by adjusting a brightness of each point of the object, and
wherein the brightness of each point is adjusted by giving a weight to a brightness of a light source such that the weight is given in proportion to an absolute value of an inner product of a direction vector at the user viewpoint and the normal vector of the mesh of the point.
2. The method of claim 1 , wherein, in the step (a), the multi-view depth and color images are images taken from all directions through an omnidirectional RGB-D camera system.
3. The method of claim 1 , wherein, in the step (b), a matching process of unifying data of the multi-view depth and color images into one coordinate system is performed to generate the point cloud data of the three-dimensional object in one coordinate system.
4. The method of claim 1 , wherein, in the step (c), the point cloud data of the three-dimensional object is sampled and converted into the mesh data of the three-dimensional object by using a Delaunay triangulation scheme.
5. A method of generating a volume hologram using a point cloud and a mesh, the method comprising:
(a) acquiring multi-view depth and color images;
(b) generating point cloud data of a three-dimensional object from the acquired multi-view depth and color images;
(c) generating mesh data of the three-dimensional object from the point cloud data of the three-dimensional object;
(d) calculating a normal vector of each mesh from the mesh data of the three-dimensional object;
(e) extracting three-dimensional data at a user viewpoint from the mesh data of the three-dimensional object by using the normal vector of the mesh; and
(f) generating hologram data from the three-dimensional data at the user viewpoint,
wherein, in the step (e), a direction vector at the user viewpoint is acquired, an inner product of the direction vector at the user viewpoint and the normal vector of the mesh is calculated to extract only a mesh where a result value of the inner product is a non-negative number, and the extracted mesh serves as the three-dimensional data at the user viewpoint, and
wherein the direction vector at the user viewpoint is acquired by setting a Z-axis direction as a basic direction vector, and rotating the basic direction vector by using a yaw, a pitch, and a roll at a rotated user viewpoint.
6. The method of claim 5 , wherein the direction vector at the user viewpoint is calculated by the following formula 1:
{right arrow over ( Z ′)}= R (θ z ) R (θ x ) R (θ y ) {right arrow over (Z)},
wherein θ z , θ x , and θ y are a yaw, a pitch, and a roll at a user viewpoint, respectively, and
Z
→
=
(
0
,
0
,
1
)
R
(
θ
z
)
=
(
cos
(
θ
z
)
-
sin
(
θ
z
)
0
sin
(
θ
z
)
cos
(
θ
z
)
0
0
0
1
)
R
(
θ
x
)
=
(
0
0
0
0
cos
(
θ
x
)
-
sin
(
θ
x
)
0
0
cos
(
θ
x
)
)
R
(
θ
y
)
=
(
cos
(
θ
y
)
0
-
sin
(
θ
y
)
0
1
0
sin
(
θ
y
)
0
cos
(
θ
y
)
)
.
7. A method of generating a volume hologram using a point cloud and a mesh, the method comprising:
(a) acquiring multi-view depth and color images;
(b) generating point cloud data of a three-dimensional object from the acquired multi-view depth and color images;
(c) generating mesh data of the three-dimensional object from the point cloud data of the three-dimensional object;
(d) calculating a normal vector of each mesh from the mesh data of the three-dimensional object;
(e) extracting three-dimensional data at a user viewpoint from the mesh data of the three-dimensional object by using the normal vector of the mesh; and
(f) generating hologram data from the three-dimensional data at the user viewpoint,
wherein, in the step (e), a direction vector at the user viewpoint is acquired, an inner product of the direction vector at the user viewpoint and the normal vector of the mesh is calculated to extract only a mesh where a result value of the inner product is a non-negative number, and the extracted mesh serves as the three-dimensional data at the user viewpoint,
wherein, in the step (f), with respect to the extracted three-dimensional data at the user viewpoint, a density of the three-dimensional data is adjusted, and the hologram data is generated from the three-dimensional data at the user viewpoint in which the density is adjusted,
wherein, with respect to mesh data corresponding to the extracted three-dimensional data at the user viewpoint, a size of each mesh is measured to determine whether the measured mesh size is less than a threshold value, the mesh is divided by adding a point to the mesh when the measured mesh size is greater than the threshold value, and the division process is repeatedly performed such that all meshes are divided to have a size less than the threshold value, thereby adjusting the density, and
wherein, with respect to a mesh having a size greater than or equal to the threshold value, a location of the point to be generated is calculated by using a circumcenter of a triangle, color information of the generated point has an average value of each point, and a normal vector is set as a normal vector of the mesh, thereby adding the point.