Granted Patent B2

US 12,361,643 · App. 18/166,148 · Granted Jul 15, 2025

Inventors: Khaled Mammou (Danville, CA); Alexandros Tourapis (Los Gatos, CA); Jungsun Kim (Sunnyvale, CA)

Assignee: Apple Inc.

G06T17/20G06T15/04G06T17/205G06V10/25G06T9/001G06T2200/04

Abstract

Claims (52)

1. A method of compressing a 3D textured mesh M(i), the 3D textured mesh M(i) being defined by connectivity C(i), geometry G(i), texture coordinates T(i), and texture connectivity CT(i), wherein the 3D textured mesh M(i) is associated with one or more 2D image attribute maps A(i) describing attributes associated with a mesh surface of the 3D textured mesh M(i), the method comprising:

pre-processing the 3D textured mesh M(i) and the attribute maps A(i) to generate a base mesh m(i) and a displacement field d(i); and

processing the 3D textured mesh M(i), the attribute maps A(i), the base mesh m(i), and the displacement field d(i) to generate a compressed bitstream b(i).

2. The method of claim 1 wherein pre-processing the 3D textured mesh M(i) and the attribute maps A(i) to generate the base mesh m(i) and the displacement field d(i) further comprises:

decimating the 3D textured mesh M(i);

subdividing the decimated 3D textured mesh to generate the base mesh m(i); and

computing the displacement field d(i) as a difference between vertices of the base mesh m(i) and the 3D textured mesh M(i).

3. The method of claim 1 wherein processing the 3D textured mesh M(i), the attribute maps A(i), the base mesh m(i), and the displacement field d(i) to generate the compressed bitstream b(i) further comprises:

quantizing the base mesh m(i);

encoding the quantized base mesh m(i) using a selected mesh encoder to produce a compressed base mesh bitstream; and

multiplexing the compressed base mesh bitstream to generate the compressed bitstream b(i).

4. The method of claim 3 wherein processing the 3D textured mesh M(i), the attribute maps A(i), the base mesh m(i), and the displacement field d(i) to generate the compressed bitstream b(i) further comprises:

decoding the compressed base mesh bitstream using a selected mesh decoder to produce a reconstructed quantized base mesh m′(i);

generating an updated displacement field d′(i) from the reconstructed quantized base mesh m′(i), the base mesh m(i), and the displacement field d(i);

performing a wavelet transform on the updated displacement field d′(i) to generate a plurality of wavelet coefficients;

quantizing the plurality of wavelet coefficients;

packing the quantized plurality of wavelet coefficients into an image sequence;

encoding the image sequence with a video encoder to generate a compressed displacement bitstream; and

multiplexing the compressed displacement bitstream with the compressed base mesh bitstream to generate the compressed bitstream b(i).

5. The method of claim 4 wherein processing the 3D textured mesh M(i), the attribute maps A(i), the base mesh m(i), and the displacement field d(i) to generate the compressed bitstream b(i) further comprises:

reconstructing the packed quantized plurality of wavelet coefficients;

unpacking, inverse quantizing, and inverse wavelet transforming the reconstructed packed quantized wavelet coefficients received from the video encoder to produce a reconstructed displacement field d″(i);

inverse quantizing the reconstructed quantized base mesh m′(i) to produce a reconstructed base mesh m″(i);

producing a reconstructed deformed mesh DM(i) from the reconstructed base mesh m″(i) and the reconstructed displacement field d″(i);

producing an updated attribute map A′(i) from the reconstructed deformed mesh DM(i), the 3D textured mesh M(i), and the attribute maps A(i);

encoding the image sequence with the video encoder to generate a compressed attribute bitstream; and

multiplexing the compressed attribute bitstream with the compressed displacement bitstream and the compressed base mesh bitstream to generate the that is multiplexed inte compressed bitstream b(i).

6. The method of claim 5 further comprising padding the updated attribute map A′(i) to allow for optimized encoding.

7. The method of claim 3 wherein the selected mesh encoder is a static mesh encoder that is determined by specification or application.

8. The method of claim 3 wherein the selected mesh encoder is a motion encoder that is determined by specification or application.

9. A method of decoding a bitstream b(i) to reconstruct a decoded deformed mesh DM(i) corresponding to a source 3D textured mesh M(i) and one or more decoded 2D image attribute maps A″(i) describing attributes associated with a mesh surface and corresponding to one or more source 2D image attribute maps A(i), the method comprising:

de-multiplexing the compressed bitstream b(i) to produce:

a compressed base mesh bitstream;

a compressed displacement bitstream; and

a compressed attribute bitstream; and

decoding the compressed base mesh bitstream, the compressed displacement bitstream, and the compressed attribute bitstream.

10. The method of claim 9 wherein decoding the compressed base mesh bitstream further comprises:

decoding the compressed base mesh bitstream using a selected mesh decoder to produce a reconstructed quantized base mesh m′(i); and

inverse quantizing the reconstructed quantized base mesh m′(i) to produce a decoded base mesh m″(i).

11. The method of claim 10 wherein decoding the compressed base mesh bitstream further comprises:

decoding the compressed displacement bitstream with a video decoder to generate resulting images, unpacking the resulting images, inverse quantizing the unpacked resulting images, and performing an inverse wavelet transform on the inverse quantized unpacked resulting images to produce a decoded displacement field d″(i); and

reconstructing the decoded deformed mesh DM(i) from the decoded base mesh m″(i) and the decoded displacement field d″(i).

12. The method of claim 11 wherein decoding the compressed attribute bitstream further comprises:

decoding the compressed attribute bitstream with the video decoder to produce a decoded attribute map A″(i).

13. The method of claim 12 further comprising postprocessing at least one of the decoded deformed mesh DM(i) and the decoded attribute map A″(i) to perform one or more functions selected from the group consisting of:

geometry smoothing;

attribute smoothing;

image or video smoothing or filtering; and

adaptive tessellation.

14. The method of claim 10 wherein the selected mesh decoder is a static mesh decoder determined by standard or application.

15. The method of claim 10 wherein the mesh decoder is a motion decoder determined by standard or application.

16. The method of claim 15 further comprising producing the reconstructed quantized base mesh m′(i) by adding a decoded motion output of the motion decoder to a decoded reference base mesh m′(j).

Assignments (1)

Continuity (8)

References Cited (116)