Motion field coding in dynamic mesh compression
A method and apparatus comprising computer code configured to cause a processor or processors to obtain a mesh sequence comprising a plurality of meshes corresponding to volumetric data of at least one three-dimensional (3D) visual content, obtain a frame of the mesh sequence corresponding to the volumetric data, the frame including a plurality of vertices of a mesh of the mesh sequence, determine a motion field comprising motion vectors of the plurality of vertices of the mesh, and encode the volumetric data based on the motion field.
1 . A method for video encoding, the method performed by at least one processor and comprising:
obtaining a mesh sequence comprising a plurality of meshes corresponding to volumetric data of at least one three-dimensional (3D) visual content;
obtaining a frame of the mesh sequence corresponding to the volumetric data, the frame comprising a plurality of vertices of a mesh of the mesh sequence;
determining a motion field comprising motion vectors of the plurality of vertices of the mesh; and
encoding the volumetric data based on the motion field and applying a one-dimensional transform to each of the motion vectors of the plurality of vertices of the mesh, and the one-dimensional transform comprises a discrete cosine transform.
2 . The method for video encoding according to claim 1 , wherein coding the volumetric data comprises:
arranging the motion vectors of the plurality of vertices of the mesh into ordered motion vectors, and
packing the ordered motion vectors into a 3-channel image.
3 . The method for video encoding according to claim 2 , wherein arranging the motion vectors of the plurality of vertices of the mesh into the ordered motion vectors is based on a predetermined order.
4 . The method for video encoding according to claim 2 , wherein channels of the 3-channel image comprise respective ones of spatial dimensions of the motion vectors.
5 . The method for video encoding according to claim 1 , wherein coding the volumetric data comprises applying a principal component analysis to the motion field.
6 . The method for video encoding according to claim 5 , wherein the principal component analysis comprises constructing a matrix comprising a number of rows, equal to a number the plurality of vertices of the mesh, and a number of columns equal to a number of spatial dimensions of the motion field.
7 . The method for video encoding according to claim 6 , wherein the principal component analysis further comprises obtaining a covariance matrix from the matrix and applying eigendecomposition to the covariance matrix.
8 . The method for video encoding according to claim 7 , wherein coding the volumetric data comprises signaling at least a plurality of eigenvalues obtained from applying eigendecomposition to the covariance matrix.
9 . The method for video encoding according to claim 8 , wherein coding the volumetric data further comprises signaling the mean of each column of the matrix along with the plurality of eigenvalues obtained from applying eigendecomposition to the covariance matrix.
10 . The method for video encoding according to claim 1 , wherein the one-dimensional transform to each of the motion vectors of the plurality of vertices of the mesh comprises obtaining a trajectory of each of the motion vectors of the plurality of vertices of the mesh, and
wherein encoding the volumetric data further comprises encoding transform coefficients resulting applying the one-dimensional transform to each of the motion vectors of the plurality of vertices of the mesh.
11 . An apparatus for video encoding, the apparatus comprising:
at least one memory configured to store computer program code;
at least one processor configured to access the computer program code and operate as instructed by the computer program code, the computer program code including:
obtaining code configured to cause the at least one processor to obtain a mesh sequence comprising a plurality of meshes corresponding to volumetric data of at least one three-dimensional (3D) visual content;
further obtaining code configured to cause the at least one processor to obtain a frame of the mesh sequence corresponding to the volumetric data, the frame comprising a plurality of vertices of a mesh of the mesh sequence;
determining code configured to cause the at least one processor to determine a motion field comprising motion vectors of the plurality of vertices of the mesh; and
encoding code configured to cause the at least one processor to encode the volumetric data based on the motion field and applying a one-dimensional transform to each of the motion vectors of the plurality of vertices of the mesh, and the one-dimensional transform comprises a discrete cosine transform.
12 . The apparatus for video encoding according to claim 11 , wherein coding the volumetric data comprises:
arranging the motion vectors of the plurality of vertices of the mesh into ordered motion vectors, and
packing the ordered motion vectors into a 3-channel image.
13 . The apparatus for video encoding according to claim 12 , wherein arranging the motion vectors of the plurality of vertices of the mesh into the ordered motion vectors is based on a predetermined order.
14 . The apparatus for video encoding according to claim 12 , wherein channels of the 3-channel image comprise respective ones of spatial dimensions of the motion vectors.
15 . The apparatus for video encoding according to claim 11 , wherein coding the volumetric data comprises applying a principal component analysis to the motion field.
16 . The apparatus for video encoding according to claim 15 , wherein the principal component analysis comprises constructing a matrix comprising a number of rows, equal to a number the plurality of vertices of the mesh, and a number of columns equal to a number of spatial dimensions of the motion field.
17 . The apparatus for video encoding according to claim 16 , wherein the principal component analysis further comprises obtaining a covariance matrix from the matrix and applying eigendecomposition to the covariance matrix.
18 . A method of processing visual media data, the method comprising:
performing a conversion between a visual media file and a bitstream of a visual media data according to a format rule indicating to:
obtain a mesh sequence comprising a plurality of meshes corresponding to volumetric data of at least one three-dimensional (3D) visual content;
obtain a frame of the mesh sequence corresponding to the volumetric data, the frame comprising a plurality of vertices of a mesh of the mesh sequence;
determine a motion field comprising motion vectors of the plurality of vertices of the mesh; and
applying a one-dimensional transform to each of the motion vectors of the plurality of vertices of the mesh, the one-dimensional transform comprises a discrete cosine transform.