Granted Patent B2

US 12,192,521 · App. 18/314,986 · Granted Jan 7, 2025

Inventors: Thuong Nguyen Canh (Palo Alto, CA); Xiaozhong Xu (State College, PA); Chao Huang (Palo Alto, CA); Xiang Zhang (Sunnyvale, CA); Shan Liu (San Jose, CA)

Assignee: TENCENT AMERICA LLC

H04N19/597H04N19/105H04N19/17

Abstract

Claims (63)

1. A method for video encoding, the method performed by at least one processor and comprising:

obtaining volumetric data of at least one three-dimensional (3D) visual content;

deriving a mesh from a frame of the volumetric data, the mesh comprising a plurality of base mesh vertices;

determining a displacement of at least one vertex, that is not of the base mesh vertices, based on a series of projections from at least one of the plurality of base mesh vertices that is a neighboring one of the plurality of base mesh vertices to the at least one vertex;

predicting the at least one vertex based at least on the determined displacement; and

encoding the volumetric data based on the predicted at least one vertex.

2. The method for video encoding according to claim 1 , wherein

encoding the volumetric data comprises encoding a scalar distance from a projected vertex to the at least one vertex.

3. The method for video encoding according to claim 2 , wherein

the series of projections comprises determining the projected vertex to be on a line from a first base mesh vertex of the plurality of base mesh vertices to a second base mesh vertex of the plurality of base mesh vertices, and

the scalar distance is normal to the line.

4. The method for video encoding according to claim 3 , wherein

the series of projections further comprises deriving a neighbor vertex as being on the line, and

determining the projected vertex to be on the line comprises applying a second scalar distance to the neighbor vertex.

5. The method for video encoding according to claim 1 , wherein the series of projections comprises:

determining a first neighboring vertex to be on a first line from a first base mesh vertex of the plurality of base mesh vertices to a second base mesh vertex of the plurality of base mesh vertices, and

determining the projected vertex to be a distance from the first neighboring vertex.

6. The method for video encoding according to claim 5 , wherein the series of projections further comprises:

determining a second neighboring vertex to be on a second line from the first base mesh vertex of the plurality of base mesh vertices to a third base mesh vertex of the plurality of base mesh vertices, and

determining the projected vertex to be the distance from both the first neighboring vertex and the second neighboring vertex.

7. The method for video encoding according to claim 6 , wherein

encoding the volumetric data comprises encoding a distance from the projected vertex to the at least one vertex.

8. The method for video encoding according to claim 6 , wherein

the plurality of base mesh vertices define a face of the mesh,

the series of projections further comprises subdividing the face into a plurality of sub-divisions and determining a nearest sub-division point of the sub-divisions to the projected vertex.

9. The method for video encoding according to claim 8 , wherein

encoding the volumetric data comprises encoding a distance from the nearest sub-division point to the at least one vertex.

10. The method for video encoding according to claim 8 , wherein

the face of the mesh comprises a quad face.

11. An apparatus for video encoding, the apparatus comprising:

at least one memory configured to store computer program code;

at least one processor configured to access the computer program code and operate as instructed by the computer program code, the computer program code including:

obtaining code configured to cause the at least one processor to obtain volumetric data of at least one three-dimensional (3D) visual content;

deriving code configured to cause the at least one processor to derive a mesh from a frame of the volumetric data, the mesh comprising a plurality of base mesh vertices;

determining code configured to cause the at least one processor to determine a displacement of at least one vertex, that is not of the base mesh vertices, based on a series of projections from at least one of the plurality of base mesh vertices that is a neighboring one of the plurality of base mesh vertices to the at least one vertex;

predicting code configured to cause the at least one processor to predict the at least one vertex based at least on the determined displacement; and

encoding code configured to cause the at least one processor to encode the volumetric data based on determining the predicted at least one vertex.

12. The apparatus for video encoding according to claim 11 , wherein

encoding the volumetric data comprises encoding a scalar distance from a projected vertex to the at least one vertex.

13. The apparatus for video encoding according to claim 12 , wherein

the series of projections comprises determining the projected vertex to be on a line from a first base mesh vertex of the plurality of base mesh vertices to a second base mesh vertex of the plurality of base mesh vertices, and

the scalar distance is normal to the line.

14. The apparatus for video encoding according to claim 13 , wherein

the series of projections further comprises deriving a neighbor vertex as being on the line, and

determining the projected vertex to be on the line comprises applying a second scalar distance to the neighbor vertex.

15. The apparatus for video encoding according to claim 11 , wherein the series of projections comprises:

determining a first neighboring vertex to be on a first line from a first base mesh vertex of the plurality of base mesh vertices to a second base mesh vertex of the plurality of base mesh vertices, and

determining the projected vertex to be a distance from the first neighboring vertex.

16. The apparatus for video encoding according to claim 15 , wherein the series of projections further comprises:

determining a second neighboring vertex to be on a second line from the first base mesh vertex of the plurality of base mesh vertices to a third base mesh vertex of the plurality of base mesh vertices, and

determining the projected vertex to be the distance from both the first neighboring vertex and the second neighboring vertex.

17. The apparatus for video encoding according to claim 16 , wherein

encoding the volumetric data comprises encoding a distance from the projected vertex to the at least one vertex.

18. The apparatus for video encoding according to claim 16 , wherein

the plurality of base mesh vertices define a face of the mesh,

the series of projections further comprises subdividing the face into a plurality of sub-divisions and determining a nearest sub-division point of the sub-divisions to the projected vertex.

19. The apparatus for video encoding according to claim 18 , wherein

encoding the volumetric data comprises encoding a distance from the nearest sub-division point to the at least one vertex.

20. A non-transitory computer readable medium storing a program causing a computer to:

obtain volumetric data of at least one three-dimensional (3D) visual content;

derive a mesh from a frame of the volumetric data, the mesh comprising a plurality of base mesh vertices;

determine a displacement of at least one vertex, that is not of the base mesh vertices, based on a series of projections from at least one of the plurality of base mesh vertices that is a neighboring one of the plurality of base mesh vertices to the at least one vertex; and

encode the volumetric data based on determining the predicted at least one vertex.

Assignments (1)

Continuity (2)

References Cited (10)