Method for encoding 3D content represented by dynamic mesh, method for decoding coded mesh bitstream of dynamic mesh representing 3D content, and system
A method for encoding three-dimensional (3D) content represented by a dynamic mesh includes: converting geometry information from a list of vertices of the dynamic mesh to a sequence of geometry component images; encoding the sequence of geometry component images of the dynamic mesh using a video encoder to generate a geometry component bitstream; decoding the geometry component bitstream to generate reconstructed geometry component images; determining a face to be removed from connectivity component images of the dynamic mesh, the face containing a vertex in the reconstructed geometry component images; updating the connectivity component images of the dynamic mesh by removing the face from the connectivity component images; encoding the updated connectivity component images to generate a connectivity component bitstream; updating, based on updating the connectivity component images, attribute component images and mapping component images of the dynamic mesh; encoding the updated attribute component images to generate an attribute component bitstream.
1 . A method for decoding a coded mesh bitstream of a dynamic mesh representing three-dimensional (3D) content, the method comprising:
reconstructing geometry information of the dynamic mesh from a geometry component bitstream in the coded mesh bitstream, the reconstructed geometry information comprising data specifying a plurality of vertices of the dynamic mesh;
reconstructing connectivity information of the dynamic mesh from a connectivity component bitstream in the coded mesh bitstream, the reconstructed connectivity information comprising data specifying a plurality of faces of the dynamic mesh;
refining the reconstructed connectivity information based on the reconstructed geometry information to generate refined connectivity information by at least dividing a face out of the plurality of faces specified by the reconstructed connectivity information into two faces based on a vertex of the plurality of vertices specified in the reconstructed geometry information;
reconstructing mapping information of the dynamic mesh from a mapping component bitstream in the coded mesh bitstream, the reconstructed mapping information comprising data specifying a list of vertex attribute coordinates in an attribute image;
refining the reconstructed mapping information based on the reconstructed geometry information to generate refined mapping information;
reconstructing the dynamic mesh based on the reconstructed geometry information, the refined connectivity information, and the refined mapping information; and
causing the reconstructed dynamic mesh to be rendered for display.
2 . The method of claim 1 , wherein refining the reconstructed connectivity information based on the reconstructed geometry information comprises:
identifying a vertex of the plurality of vertices that is located inside a face of the plurality of faces;
determining a projected vertex of the vertex on an edge of the face; and
dividing the face into two refined faces, each refined face has the projected vertex as one vertex.
3 . The method of claim 2 , wherein determining the projected vertex of the vertex comprises:
determining a nearest edge for the vertex among edges of the face, wherein a distance between the vertex to the nearest edge is smaller than another distance between the vertex to another edge of the edges of the face; and
determining the projected vertex as an intersection of a line with the nearest edge, the line passing through the vertex and an opposite vertex of the face that does not belong to the nearest edge.
4 . The method of claim 2 , wherein refining the reconstructed mapping information comprises:
determining a transformation based on the vertex and the projected vertex; and
applying the transformation to vertex attribute coordinates in the list of the vertex attribute coordinates that correspond to the vertex.
5 . The method of claim 4 , wherein the transformation comprises an affine transformation.
6 . The method of claim 2 , wherein each of the two refined faces have a same orientation as the face.
7 . The method of claim 1 , wherein refining the reconstructed connectivity information based on the reconstructed geometry information to generate refined connectivity information comprises dividing a face out of the plurality of faces specified by the reconstructed connectivity information into two faces based on a vertex of the plurality of vertices specified in the reconstructed geometry information.
8 . A system comprising:
a processor; and
a non-transitory computer-readable medium communicatively coupled to the processor, wherein the processor is configured to execute program code stored in the non-transitory computer-readable medium and thereby perform operations comprising:
reconstructing geometry information of a dynamic mesh from a geometry component bitstream in a coded mesh bitstream of the dynamic mesh, the reconstructed geometry information comprising data specifying a plurality of vertices of the dynamic mesh;
reconstructing connectivity information of the dynamic mesh from a connectivity component bitstream in the coded mesh bitstream, the reconstructed connectivity information comprising data specifying a plurality of faces of the dynamic mesh;
refining the reconstructed connectivity information based on the reconstructed geometry information to generate refined connectivity information by at least dividing a face out of the plurality of faces specified by the reconstructed connectivity information into two faces based on a vertex of the plurality of vertices specified in the reconstructed geometry information;
reconstructing mapping information of the dynamic mesh from a mapping component bitstream in the coded mesh bitstream, the reconstructed mapping information comprising data specifying a list of vertex attribute coordinates in an attribute image;
refining the reconstructed mapping information based on the reconstructed geometry information to generate refined mapping information;
reconstructing the dynamic mesh based on the reconstructed geometry information, the refined connectivity information, and the refined mapping information; and
causing the reconstructed dynamic mesh to be rendered for display.
9 . The system of claim 8 , wherein refining the reconstructed connectivity information based on the reconstructed geometry information comprises:
identifying the vertex of the plurality of vertices that is located inside the face of the plurality of faces;
determining a projected vertex of the identified vertex on an edge of the face; and
dividing the face into two refined faces, each refined face has the projected vertex as one vertex.
10 . The system of claim 9 , wherein determining the projected vertex of the vertex comprises:
determining a nearest edge for the vertex among edges of the face, wherein a distance between the vertex to the nearest edge is smaller than another distance between the vertex to another edge of the edges of the face; and
determining the projected vertex as an intersection of a line with the nearest edge, the line passing through the vertex and an opposite vertex of the face that does not belong to the nearest edge.
11 . The system of claim 9 , wherein refining the reconstructed mapping information comprises:
determining a transformation based on the vertex and the projected vertex; and
applying the transformation to vertex attribute coordinates in the list of the vertex attribute coordinates that correspond to the vertex.
12 . The system of claim 11 , wherein the transformation comprises an affine transformation.
13 . The system of claim 9 , wherein each of the two refined faces have a same orientation as the face.