Method for decoding immersive video and method for encoding immersive video
An image encoding method according to the present disclosure may include classifying a plurality of view images into a basic image and an additional image; performing pruning for at least one of the plurality of view images based on a result of the classification; generating an atlas based on a result of performing the pruning; and encoding the atlas and metadata for the atlas. In this case, the metadata may include spherical harmonic function information on a point in a three-dimensional space.
1 . An image encoding method comprising:
classifying a plurality of view images into a basic view and an additional view;
performing pruning for at least one of the plurality of view images based on a result of the classification;
generating an atlas based on the result of performing the pruning; and
encoding the atlas and metadata for the atlas,
wherein the metadata includes spherical harmonic function information on a point in a three-dimensional space.
2 . The method according to claim 1 , wherein:
the spherical harmonic function information includes a coefficient of the spherical harmonic function, and
the coefficient is determined based on a color value unprojected on the point from at least one of the plurality of view images.
3 . The method according to claim 1 , wherein:
information on the spherical harmonic function is encoded as data on a patch in the atlas.
4 . The method according to claim 3 , wherein:
for the patch, a flag representing whether the point that the spherical harmonic function is allocated to the patch exists is encoded, and
for a case in which the point that the spherical harmonic function is allocated to the patch exists, information on the spherical harmonic function is encoded.
5 . The method according to claim 1 , wherein:
information on the spherical harmonic function is encoded only for the basic image among the plurality of view images.
6 . The method according to claim 1 , wherein:
for each of regions partitioned by a quadtree structure, information representing whether the point to which the spherical harmonic function is allocated exists is encoded, and
for a case in which the point to which the spherical harmonic function is allocated exists in the partitioned region, information on the spherical harmonic function for the point in the partitioned region is encoded.
7 . The method according to claim 1 , wherein:
for each of points forming a grid structure in the three-dimensional space, the spherical harmonic function is encoded.
8 . The method according to claim 1 , wherein:
only for the point positioned on a non-Lambertian surface among points forming a grid structure in the three-dimensional space, information on the spherical harmonic function is encoded.
9 . The method according to claim 1 , wherein:
a plurality of closed points are configured on an object surface searched based on a depth map for the basic image and information on the spherical harmonic function is encoded for each of the plurality of closed points.
10 . The method according to claim 9 , wherein:
a second layer separated by an offset from a first layer configured with the closed points is configured, and
information on the spherical harmonic function is additionally encoded for each of open points included in the second layer.
11 . The method according to claim 9 , wherein:
view dependence information of the additional view not overlapped with the basic image is extracted as a patch, and
information of the spherical harmonic function is encoded for the point included in the patch.
12 . The method according to claim 1 , wherein:
information of the spherical harmonic function includes at least one of a scale of the point, an initial reference coordinate of the spherical harmonic function, a number of coefficients of the spherical harmonic function or a value of each of coefficients.
13 . An image decoding method comprising:
decoding an atlas and metadata for the atlas; and
generating a viewport image by using the atlas and the metadata,
wherein the metadata includes spherical harmonic function information on a reference point in a three-dimensional space.
14 . The method according to claim 13 , wherein:
a pixel value for a first point in the viewport image is derived based on the spherical harmonic function for a plurality of reference points adjacent to the first point.
15 . The method according to claim 14 , wherein:
the plurality of reference points configure a unit grid that the first point is included.
16 . The method according to claim 13 , wherein:
information on the spherical harmonic function is included in data on a patch in the atlas.
17 . The method according to claim 16 , wherein:
a flag representing whether a point that the spherical harmonic function is allocated to the patch exists is decoded, and
when the flag indicates that the point that the spherical harmonic function is allocated to the patch exists, information on the spherical harmonic function is decoded.
18 . The method according to claim 13 , wherein:
the metadata includes information on the spherical harmonic function for reference points belonging to a first layer and reference points belonging to a second layer, respectively.
19 . The method according to claim 18 , wherein:
the metadata includes offset information representing a position difference between the first layer and the second layer.
20 . A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
classifying a plurality of view images into a basic image and an additional image;
performing pruning for at least one of the plurality of view images based on a result of the classification;
generating an atlas based on the result of performing the pruning; and
encoding the atlas and metadata for the atlas,
wherein metadata includes spherical harmonic function information on a point in a three-dimensional space.