Systems and methods compression, transfer, and reconstruction of three-dimensional (3D) data meshes

1. A method comprising:

receiving a plurality of time-synchronized video streams of a subject from video cameras at known vantage points;

generating, based on the plurality of time-synchronized video streams, a three-dimensional (3D) mesh of the subject based on frames of the time-synchronized video streams associated with a first time;

obtaining a facial-mesh model;

generating a transformed facial-mesh model based on a facial portion of the 3D mesh of the subject and the facial-mesh model;

generating a hybrid mesh by combining the transformed facial-mesh model and at least a portion of the 3D mesh;

generating a current 3D mesh of the subject based on frames of the time-synchronized video streams associated with a second time that temporally follows the first time;

generating a deformed historical 3D mesh by applying a non-rigid deformation process to the hybrid mesh that deforms the hybrid mesh based on the current 3D mesh;

compressing the deformed historical 3D mesh to form at least one triangle-based 3D submesh of the subject, wherein the triangle-based 3D submesh includes a plurality of submesh vertices that define a plurality of submesh triangles;

identifying strips of the submesh triangles;

generating triangle-strip data representing the identified strips of submesh triangles;

generating compressed-submesh data that includes the triangle-strip data; and

transmitting the compressed-submesh data to a receiver for reconstruction of the triangle-based 3D submesh of the subject.

2. The method of claim 1 , further comprising reducing a triangle density of one or more subparts of the triangle-based 3D submesh prior to identifying the strips of the submesh triangles.

3. The method of claim 2 , wherein the one or more subparts include a head portion of the subject.

4. The method of claim 2 , wherein the one or more subparts include a hands portion of the subject.

5. The method of claim 2 , wherein the reducing the triangle density of one or more subparts of the triangle-based 3D submesh further comprises:

generating a modified subpart by merging nearby vertices subparts; and

removing any resulting zero-area triangles from the modified subpart.

6. The method of claim 1 , wherein the transformed facial-mesh model is generated in response to aggregated error differences between feature points on the facial-mesh model and corresponding feature points on the facial portion.

7. The method of claim 1 , further comprising reducing a triangle density of the triangle-based 3D submesh prior to identifying the strips of the submesh triangles.

8. The method of claim 7 , wherein reducing the triangle density of the triangle-based 3D submesh prior to identifying the strips of the submesh triangles further comprises:

generating a modified subpart by merging nearby vertices subparts; and

removing any resulting zero-area triangles from the modified subpart.

9. The method of claim 1 , wherein generating triangle-strip data representing one of the identified strips of the submesh triangles includes:

defining a first triangle in a given strip using three vertices; and

defining each subsequent triangle in the given strip using a single vertex.

10. The method of claim 9 , wherein each vertex is defined in the triangle-strip data by at least one or more of:

a first floating-point value for an x-coordinate of the vertex;

a second floating-point value for a y-coordinate of the vertex; and

a third floating-point value for a z-coordinate of the vertex.

11. The method of claim 10 , wherein each such vertex is defined in the triangle-strip data with:

a first step-count value for an x-coordinate of the vertex;

a second step-count value for a y-coordinate of the vertex; and

a third step-count value for a z-coordinate of the vertex.

12. The method of claim 9 , wherein a first vertex of the first triangle is defined in the triangle-strip data by one or more of:

a first from-origin value for an x-coordinate of the vertex;

a second from-origin value for a y-coordinate of the vertex; and

a third from-origin value for a z-coordinate of the vertex; and

each subsequent vertex in the given strip is defined in the triangle-strip data with one or more of:

an x-coordinate delta from the x-coordinate of a preceding vertex in the given strip;

a y-coordinate delta from the y-coordinate of the preceding vertex in the given strip; and

a z-coordinate delta from the z-coordinate of the preceding vertex in the given strip.

13. The method of claim 12 , wherein each from-origin value and each coordinate delta is expressed in step counts.

14. The method of claim 13 , wherein:

each from-origin value is a 10-bit step-count value; and

each coordinate delta is a 4-bit step-count value.

15. The method of claim 13 , further comprising using a reduced bit count to abbreviate at least one frequently occurring coordinate delta.

16. The method of claim 15 , wherein using the reduced bit count comprises using Huffman encoding.

17. The method of claim 1 , further comprising generating and sending to the receiver one or more duplicate-vertex lists.

18. The method of claim 17 , wherein sending to the receiver the one or more duplicate-vertex lists includes sending the one or more duplicate-vertex lists to the receiver during intermediate time frames.

19. A submesh-compression system comprising:

a communication interface;

a processor; and

data storage containing instructions executable by the processor for causing a presenter server system to carry out a set of functions, the set of functions including:

receiving a plurality of time-synchronized video streams of a subject from video cameras at known vantage points;

generating, based on the plurality of time-synchronized video streams, a three-dimensional (3D) mesh of the subject based on frames of the time-synchronized video streams associated with a first time;

obtaining a facial-mesh model;

generating a transformed facial-mesh model based on a facial portion of the 3D mesh of the subject and the facial-mesh model;

generating a hybrid mesh by combining the transformed facial-mesh model and at least a portion of the 3D mesh;

generating a current 3D mesh of the subject based on frames of the time-synchronized video streams associated with a second time that temporally follows the first time;

generating a deformed historical 3D mesh by applying a non-rigid deformation process to the hybrid mesh that deforms the hybrid mesh based on the current 3D mesh;

compressing the deformed historical 3D mesh to form a triangle-based 3D submesh of the subject, the triangle-based 3D submesh including a plurality of submesh vertices that define a plurality of submesh triangles;

identifying strips of the submesh triangles;

generating triangle-strip data representing the identified strips of the submesh triangles;

generating compressed-submesh data that includes the triangle-strip data; and

transmitting the compressed-submesh data to a receiver for reconstruction of the triangle-based 3D submesh of the subject.

20. A non-transitory computer program product comprising computer-executable code that is operable, when executed by a processor, to cause the processor to execute steps comprising:

receiving a plurality of time-synchronized video streams of a subject from video cameras at known vantage points;

generating, based on the plurality of time-synchronized video streams, a three-dimensional (3D) mesh of the subject based on frames of the time-synchronized video streams associated with a first time;

obtaining a facial-mesh model;

generating a transformed facial-mesh model based on a facial portion of the 3D mesh of the subject and the facial-mesh model;

generating a hybrid mesh by combining the transformed facial-mesh model and at least a portion of the 3D mesh;

generating a current 3D mesh of the subject based on frames of the time-synchronized video streams associated with a second time that temporally follows the first time;

generating a deformed historical 3D mesh by applying a non-rigid deformation process to the hybrid mesh that deforms the hybrid mesh based on the current 3D mesh;

compressing the deformed historical 3D mesh to form at least one triangle-based 3D submesh of the subject, wherein the triangle-based 3D submesh includes a plurality of submesh vertices that define a plurality of submesh triangles;

identifying strips of the submesh triangles;

generating triangle-strip data representing the identified strips of the submesh triangles;

generating compressed-submesh data that includes the triangle-strip data; and

transmitting the compressed-submesh data to a receiver for reconstruction of the triangle-based 3D submesh of the subject.