IP Library › Granted Patent US 12,260,597
Granted Patent B2
US 12,260,597 · App. 18/081,050 · Granted Mar 25, 2025

Methods and apparatus for point cloud compression bitstream format

Inventors: Ahmed Hamza (Montreal, CA); Yong He (San Diego, CA)
Assignee: InterDigital VC Holdings, Inc.
G06T9/00H04N19/44H04N19/70G06T15/00G06T15/08G06T15/10
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,260,597
App. No.
18/081,050
Granted
Mar 25, 2025
Kind
B2
Abstract

Methods, apparatus, systems, architectures and interfaces for encoding and/or decoding point cloud bitstreams including coded point cloud sequences are provided. Included among such methods, apparatuses, systems, architectures, and interfaces is an apparatus that may include a processor and memory. A method may include any of: mapping components of the point cloud bitstream into tracks; generating information identifying any of geometry streams or texture streams according to the mapping of the components; generating information associated with layers corresponding to respective geometry component streams; and generating information indicating operation points associated with the point cloud bitstream.

Claims (39)

1. A method of conveying decoding information for a point cloud (PC) bitstream of coded PC sequences, the method comprising:

mapping a metadata component bitstream of the PC bitstream into a first restricted video track of an International Organization for Standardization/International Electrotechnical Commission Base Media File Format (ISOBMFF), wherein the first restricted video track of the ISOBMFF comprises one or more access units of the metadata component bitstream;

mapping a geometry sub-bitstream of the PC bitstream into a second restricted video track of the ISOBMFF, wherein the second restricted video track of the ISOBMFF comprises one or more access units of the geometry sub-bitstream;

mapping an occupancy sub-bitstream of the PC bitstream into a third restricted video track of the ISOBMFF, wherein the third restricted video track of the ISOBMFF comprises one or more access units of the occupancy sub-bitstream;

adding, to the first restricted video track, indications of links between the metadata component bitstream and the geometry sub-bitstream and between the metadata component bitstream and the occupancy sub-bitstream using a track reference tool; and

generating an ISOBMFF container to transmit the metadata component bitstream mapping, the geometry sub-bitstream mapping, and the occupancy sub-bitstream mapping for the PC bitstream.

2. The method of claim 1 , further comprising adding a restricted scheme information box to the first restricted video track, the second restricted video track, and the third restricted video track, the restricted scheme information box indicating that the respective video track is a restricted video track.

3. The method of claim 1 , wherein the second restricted video track and the third restricted video track comprise an indication of one or more layers of the PC bitstream.

4. The method of claim 3 , wherein each of the one or more layers is associated with a respective depth with respect to a depth image plane.

5. The method of claim 1 , wherein the first restricted video track comprises a plurality of samples associated with the metadata component bitstream of the PC bitstream, the second restricted video track comprises a plurality of second samples associated with the geometry sub-bitstream of the PC bitstream, and the third restricted video track comprises a plurality of third samples associated with the occupancy sub-bitstream of the PC bitstream.

6. The method of claim 1 , wherein the first restricted video track, the second restricted video track, and the third restricted video track in one coded PC sequence are signaled in a single location of the ISOBMFF container.

7. The method of claim 1 , further comprising mapping an attribute sub-bitstream of the PC bitstream into a fourth restricted video track of the ISOBMFF, wherein the fourth restricted video track of the ISOBMFF comprises one or more access units of the attribute sub-bitstream.

8. The method of claim 7 , wherein the attribute sub-bitstream is associated with an attribute type that comprises a color, a transparency, a time of acquisition, or another material property.

9. The method of claim 1 , further comprising generating one or more timed-metadata component bitstreams,

wherein samples associated with the timed-metadata component bitstreams are included in a MediaDataBox.

10. The method of claim 1 , wherein the metadata component bitstream comprises a first reference to the geometry sub-bitstream and a second reference to the occupancy sub-bitstream.

11. An apparatus comprising circuitry for conveying decoding information for a point cloud (PC) bitstream of coded PC sequences the circuitry including any of a transmitter, a receiver, a processor and memory, configured to:

map a metadata component bitstream of the PC bitstream into a first restricted video track of an International Organization for Standardization/International Electrotechnical Commission Base Media File Format (ISOBMFF), wherein the first restricted video track of the ISOBMFF comprises one or more access units of the metadata component bitstream;

map a geometry sub-bitstream of the PC bitstream into a second restricted video track of the ISOBMFF, wherein the second restricted video track of the ISOBMFF comprises one or more access units of the geometry sub-bitstream;

map an occupancy sub-bitstream of the PC bitstream into a third restricted video track of the ISOBMFF, wherein the third restricted video track of the ISOBMFF comprises one or more access units of the occupancy sub-bitstream;

add, to the first restricted video track, indications of links between the metadata component bitstream and the geometry sub-bitstream and between the metadata component bitstream and the occupancy sub-bitstream using a track reference tool; and

generate an ISOBMFF container to transmit the metadata component bitstream mapping, the geometry sub-bitstream mapping, and the occupancy sub-bitstream mapping for the PC bitstream.

12. The apparatus of claim 11 , wherein the circuitry is further configured to add a restricted scheme information box to the first restricted video track, the second restricted video track, and the third restricted video track, the restricted scheme information box indicating that the respective video track is a restricted video track.

13. The apparatus of claim 11 , wherein the first restricted video track comprises a plurality of samples associated with the metadata component bitstream of the PC bitstream, the second restricted video track comprises a plurality of second samples associated with the geometry sub-bitstream of the PC bitstream, and the third restricted video track comprises a plurality of third samples associated with the occupancy sub-bitstream of the PC bitstream, and

wherein the second restricted video track and the third restricted video track comprise an indication of one or more layers of the PC bitstream.

14. The apparatus of claim 13 , wherein each of the one or more layers is associated with a respective depth with respect to a depth image plane.

15. The apparatus of claim 11 , wherein the metadata component bitstream comprises a first reference to the geometry sub-bitstream and a second reference to the occupancy sub-bitstream.

16. The apparatus of claim 11 , wherein the first restricted video track, the second restricted video track, and the third restricted video track in one coded PC sequence are signaled in a single location of the ISOBMFF container.

17. The apparatus of claim 11 , wherein the circuitry is further configured to map an attribute sub-bitstream of the PC bitstream into a fourth restricted video track of the ISOBMFF, wherein the fourth restricted video track of the ISOBMFF comprises one or more access units of the attribute sub-bitstream.

18. The apparatus of claim 17 , wherein the attribute sub-bitstream is associated with an attribute type that comprises a color, a transparency, a time of acquisition, or another material property.

19. The apparatus of claim 11 , wherein the circuitry is further configured to generate one or more timed-metadata component bitstreams,

wherein samples associated with the timed-metadata component bitstreams are included in a MediaDataBox.

20. An apparatus comprising circuitry for decoding a point cloud (PC) bitstream of coded PC sequences, the circuitry including any of a transmitter, a receiver, a processor and memory, configured to:

receive an International Organization for Standardization/International Electrotechnical Commission Base Media File Format (ISOBMFF) container that comprises:

a first restricted video track of an ISOBMFF that comprises one or more access units of a metadata component bitstream of the PC bitstream,

a second restricted video track of the ISOBMFF that comprises one or more access units of a geometry sub-bitstream of the PC bitstream, and

a third restricted video track of the ISOBMFF that comprises one or more access units of an occupancy sub-bitstream of the PC bitstream,

wherein the first restricted video track comprises indications of links between the metadata component bitstream and the geometry sub-bitstream and between the metadata component bitstream and the occupancy sub-bitstream; and

decode the first restricted video track, the second restricted video track, and the third restricted video track of the ISOBMFF container to display the PC bitstream.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 11, 2024
From: VID SCALE, INC.
To: INTERDIGITAL VC HOLDINGS, INC.
Reel/Frame 068284/0031 →
Continuity (4)
Continuation 17277064
Provisional Application 62820026 · Mar 18, 2019
Provisional Application 62732774 · Sep 18, 2018
Related Publication 20230119983A1 · Apr 20, 2023
References Cited (41)
US 11095908B2 · Dawar · 2021 [cited by examiner]
US 11132818B2 · Mammou · 2021 [cited by examiner]
US 11138762B2 · Sinharoy · 2021 [cited by examiner]
US 11568573B2 · Hamza et al. · 2023 [cited by applicant]
US 20150304665A1 · Hannuksela et al. · 2015 [cited by applicant]
US 20160373771A1 · Hendry · 2016 [cited by examiner]
US 20170347055A1 · Dore et al. · 2017 [cited by applicant]
US 20180268570A1 · Budagavi et al. · 2018 [cited by applicant]
US 20200112709A1 · Lim · 2020 [cited by applicant]
US 20210152808A1 · He · 2021 [cited by examiner]
US 20210235056A1 · Takahashi · 2021 [cited by examiner]
US 20220053205A1 · Katsumata et al. · 2022 [cited by applicant]
CN 106105220A · 2016 [cited by applicant]
CN 107426559A · 2017 [cited by applicant]
EP 3310052A1 · 2018 [cited by applicant]
EP 3349182A1 · 2018 [cited by applicant]
JP 2020515937A · 2020 [cited by applicant]
WO WO2018130491A1 · 2018 [cited by applicant]
WO WO2020008758A1 · 2020 [cited by applicant]
WO WO2020060813A1 · 2020 [cited by applicant]
WO WO2020137854A1 · 2020 [cited by applicant]
PCC Category 2 bitstream structure using ISOBMFF movie fragment, m43622, MPEG 123; Ljubljana, Slovenia, Jul. 018. [cited by applicant]
Information technology—Coding of audio-visual objects—Part 15: Carriage of network abstraction layer (NAL) unit Structured video in the ISO base media file format, ISO/IEC 14496-15:2021(E), 282 pages. [cited by applicant]
Mammou et al., Study Text of ISO/IEC CD 23090 5: video-based Point Cloud Compression, ISO/IEC JTC1/SC29/ ING11 N18180, Jan. 2019, Marrakesh, Morocco, 103 pages. [cited by applicant]
Coding of audio-visual objects, Part 15: Carriage of network abstraction layer (NAL) unit structured video in the ISO base media file format, Reference No. ISO/IEC 14496-15:2017(E); Fourth edition, Feb. 2017. [cited by applicant]
Ksu, Emre B., (Nokia) et al.; “[PCCJ On high level syntax of PCC”, MPEG Meeting, Motion Picture Expert Group or SO/IEC JTC1/SC29/WG11, No. M43580, Jul. 11, 2018, Ljubljana, Slovenia, 4 pages. [cited by applicant]
“Information technology—Coding of audio-visual objects”, Part 12: ISO base media file format, ISO/IEC 14496-12 4th edition 2012. [cited by applicant]
Takahashi, Ryohei (Sony), et al. :“Clarification on PCC category 2 bitstream from system technology aspect”, MPEG Meeting, Motion Picture Expert Group or ISO/IEC JTC1/SC29/WG11, No. m43491, Jul. 11, 2018, 2 pages. [cited by applicant]
Hamza, Ahmed (Interdigital), et al. : “Update to Questions Related to V-PCC File Format Design”, MPEG Meeting, Motion Picture Expert Group or ISO/IEC JTC1/SC29/WG11, No. m49118, Jul. 7, 2019, Gothenburg, SE, 5 pages. [cited by applicant]
Working draft of Point Cloud Coding for Category 2 (Draft 1), ISO/IEC JTC1/SC29/WG11, MPEG 2018, No. 17534, Apr. 2018, San Diego, US, 33 pages. [cited by applicant]
Yip, Eric, {SAMSUNG) et al. : “WO of Storage of V-PCC in ISOBMFF Files”, MPEG Meeting, Motion Picture Expert Group or ISO/IEC JTC1/SC29/WG11 MPEG2018, No. m45204, Oct. 11, 2018, Macau, Macao SAR., China, 7 pages. [cited by applicant]
CE2.10 on PCC Metadata, m43720, MPEG 123; Ljubljana, Slovenia, Jul. 2018. [cited by applicant]
Coding of audio-visual objects, Part 12: ISO base media file format, Dec. 15, 2015, 248 pages. [cited by applicant]
Hamza. Ahmed, (Interdigital) et al. : “On Carriage of V-PCC Layers in ISOBMFF”, MPEG Meeting, Motion Picture Expert Group or ISO/IEC JTC1/SC29/WG11, No. m49105, Jul. 3, 2019, Gothenburg, SE, 4 pages. [cited by applicant]
“First idea on Systems technologies for Point Cloud Coding”; ISO/IEC JTC1/SC29/WG11; MPEG2018, No. N17675; Apr. 2018, San Diego. [cited by applicant]
Use Cases for Point Cloud Compression (PCC), N16331, MPEG 115, Jun. 2016. [cited by applicant]
Hannuksela, Miska (Nokia), et al. : “The stream access-point sample grouping”, ISO/IEC JTC1/SC29/WG11, MPEG2014, No. M33155, Mar. 2014, Valencia, Spain, 4 pages. [cited by applicant]
Skupin, R., et al., “Standardization Status of 360 Degree Video Coding and Delivery”, IEEE, Visual Communications and Image Processing (VCIP), 2017, 4 pages. [cited by applicant]
Miska M Hannuksela et al, “SAP spectext 04 (W12640-pt12-4th-R11).docx Integrated text of ISO/IEC 14496-12:201X(E) E11 Technology—Coding of Audio-Visual Objects—Part 12: ISO base media file format” Annex to “A Stream Acc… [cited by applicant]
Aksu, Emre B., (Nokia), et al., “PCCJ On high level syntax of PCC”, MPEG Meeting, Motion Picture Expert Group or SO/IEC JTC1/SC29/WG11, No. M43580, Ljubljana, Slovenia, Jul. 11, 2018, 4 pages. [cited by applicant]
Yip, Eric, et al., “WD of Storage of V-PCC in ISOBMFF File”, MPEG Meeting, Motion Picture Expert Group or ISO/IEC JTC1/SC29/WG11 MPEG2018, No. m45204, Macau, Macao SAR., China, Oct. 11, 2018, 7 pages. [cited by applicant]