V3C or other video-based coding patch correction vector determination, signaling, and usage

1 . An apparatus, comprising:

one or more processors; and

one or more memories storing instructions that, when executed by the one or more processors, cause the apparatus at least to perform:

obtaining by the apparatus a mesh representation of a three-dimensional object;

performing by the apparatus an encoding process comprising:

segmenting the three-dimensional object into corresponding segmented two-dimensional projections;

encoding the segmented two-dimensional projections into a bitstream;

reconstructing a three-dimensional mesh from the segmented and encoded two-dimensional projections; and

deriving an error vector, describing a shift between three-dimensional data, from patches reconstructed from corresponding encoded patches of the bitstream, and original data in the mesh representation, the patches being submeshes of the reconstructed three-dimensional mesh, wherein deriving the error vector comprises minimizing a three-dimensional error function of the reconstructed three-dimensional mesh and an original mesh representation of the three-dimensional object;

signaling by the apparatus the error vector in or along with the bitstream; and

transmitting by the apparatus the bitstream and the signaled error vector.

2 . The apparatus according to claim 1 , wherein the shift is described via location correction information, rotation correction information, or both location correction information and rotation correction information.

3 . The apparatus according to claim 1 , wherein deriving the error vector comprises summing any rounding and quantization errors introduced during one or more of patch creation, patch projection, patch packing, or patch metadata creation during the encoding, to form an overall error in a three-dimensional space to which the three-dimensional object belongs.

4 . The apparatus according to claim 1 , wherein deriving the error vector comprises deriving the error vector by a combination of minimizing an error function, and summing quantization and rounding errors in a processing chain for the reconstructing.

5 . The apparatus according to claim 1 , wherein the one or more memories further store instructions that, when executed by the one or more processors, cause the apparatus at least to perform: signaling, in or along with the bitstream, indication of precision, range or precision and range corresponding to patches of the error vector.

6 . The apparatus according to claim 1 , wherein signaling using patch data units or atlas parameter sets is provided in supplemental enhancement information (SEI) messages.

7 . The apparatus according to claim 2 , wherein:

the reconstructing is benefited by increasing precision of three-dimensional offset offsets; and

the one or more memories further store instructions that, when executed by the one or more processors, cause the apparatus at least to perform: signaling, in or along with the bitstream, precision of three-dimensional offsets per sequence, frame, tile, or patch.

8 . The apparatus according to claim 1 , wherein the one or more memories further store instructions that, when executed by the one or more processors, cause the apparatus at least to perform: signaling, in or along with the bitstream, information for patch offsets for all reconstructed axes at least by adding a flag in a sequence or frame parameter set for indicating presence of offsets for axes being used and information for reconstructed axes quantizers, the signaling performed using a sequence or frame parameter set or atlas tile header or patch data unit.

9 . The apparatus according to claim 1 , wherein the one or more memories further store instructions that, when executed by the one or more processors, cause the apparatus at least to perform: signaling, in or along with the bitstream, that a precision quantizer is enabled per patch at least by a defined table of common quantizer values, and adding an index to the defined table in corresponding impacted patch data units.

10 . The apparatus according to claim 1 , wherein the one or more memories further store instructions that, when executed by the one or more processors, cause the apparatus at least to perform: signaling, in or along with the bitstream, a common precision quantizer for all patch level offsets, along with a flag indicating presence of the common precision quantizer.

11 . An apparatus, comprising:

one or more processors; and

one or more memories storing instructions that, when executed by the one or more processors, cause the apparatus at least to perform:

receiving by the apparatus a bitstream and signaled patch correction vector elements in an error vector in or along with the bitstream, the bitstream comprising two-dimensional projections of a mesh representation of a three-dimensional object, the receiving further comprising receiving indication of precision, range, or precision and range corresponding to patches of the error vector;

performing by the apparatus a decoding process on the bitstream to at least form decoded two-dimensional projections of the mesh representation;

reconstructing by the apparatus a three-dimensional mesh from decoded two-dimensional projections;

applying by the apparatus the patch correction vector elements from the error vector either in the decoding process or the reconstructing, and modifying the precision, the range, or the precision and range for corresponding patches of the error vector during the applying;

producing by the apparatus a modified three-dimensional mesh, as an output mesh, based at least on the application of the patch correction vector elements; and

outputting by the apparatus the output mesh.

12 . The apparatus according to claim 11 , wherein the signaled patch correction vector elements comprise location correction information, rotation correction information, or both location correction information and rotation correction information.

13 . The apparatus according to claim 11 , wherein:

the receiving further receiving signaling, in or along with the bitstream, that a precision quantizer is enabled per patch at least by a defined table of common quantizer values and an index added to the defined table in corresponding impacted patch data units; and

the applying by the apparatus the patch correction vector elements further comprises using the defined table of the common quantizer values and the index during the applying.

14 . The apparatus according to claim 11 , wherein:

receiving further comprising receiving signaling, in or along with the bitstream, of a common precision quantizer for all patch level offsets, along with a flag indicating presence of the common precision quantizer; and

the applying by the apparatus the patch correction vector elements further comprises using, in response to the flag indicating presence of the common precision quantizer, the common precision quantizer during the applying.

15 . An apparatus, comprising:

one or more processors; and

one or more memories storing instructions that, when executed by the one or more processors, cause the apparatus at least to perform:

receiving by the apparatus a bitstream and signaled patch correction vector elements in an error vector in or along with the bitstream, the bitstream comprising two-dimensional projections of a mesh representation of a three-dimensional object, wherein signaling using patch data units or atlas parameter sets is provided in supplemental enhancement information (SEI) messages;

performing by the apparatus a decoding process on the bitstream to at least form decoded two-dimensional projections of the mesh representation;

reconstructing by the apparatus a three-dimensional mesh from decoded two-dimensional projections;

applying by the apparatus the patch correction vector elements from the error vector either in the decoding process or the reconstructing;

producing by the apparatus a modified three-dimensional mesh, as an output mesh, based at least on the application of the patch correction vector elements; and

outputting by the apparatus the output mesh.

16 . An apparatus, comprising:

one or more processors; and

one or more memories storing instructions that, when executed by the one or more processors, cause the apparatus at least to perform:

receiving by the apparatus a bitstream and signaled patch correction vector elements in an error vector in or along with the bitstream, the bitstream comprising two-dimensional projections of a mesh representation of a three-dimensional object, the receiving further comprises receiving signaling, in or along with the bitstream, precision of three-dimensional offsets per sequence, frame, tile, or patch;

performing by the apparatus a decoding process on the bitstream to at least form decoded two-dimensional projections of the mesh representation;

reconstructing by the apparatus a three-dimensional mesh from decoded two-dimensional projections;

applying by the apparatus the patch correction vector elements from the error vector either in the decoding process or the reconstructing, the applying by the apparatus the patch correction vector elements further comprises modifying the precision of three-dimensional offsets per sequence, frame, tile, or patch during the applying;

producing by the apparatus a modified three-dimensional mesh, as an output mesh, based at least on the application of the patch correction vector elements; and

outputting by the apparatus the output mesh.

17 . An apparatus, comprising:

one or more processors; and

one or more memories storing instructions that, when executed by the one or more processors, cause the apparatus at least to perform:

receiving by the apparatus a bitstream and signaled patch correction vector elements in an error vector in or along with the bitstream, the bitstream comprising two-dimensional projections of a mesh representation of a three-dimensional object, the receiving further comprising receiving, in or along with the bitstream, information for patch offsets for all reconstructed axes at least by adding a flag in a sequence or frame parameter set for indicating presence of offsets for axes being used and information for reconstructed axes quantizers, the signaling performed using a sequence or frame parameter set or atlas tile header or patch data unit;

performing by the apparatus a decoding process on the bitstream to at least form decoded two-dimensional projections of the mesh representation;

reconstructing by the apparatus a three-dimensional mesh from decoded two-dimensional projections;

applying by the apparatus the patch correction vector elements from the error vector either in the decoding process or the reconstructing, the applying by the apparatus the patch correction vector elements further comprises using the offsets, in response to the flag indicating presence of the offsets, during the applying;

producing by the apparatus a modified three-dimensional mesh, as an output mesh, based at least on the application of the patch correction vector elements; and

outputting by the apparatus the output mesh.