IP Library › Granted Patent US 12,256,096
Granted Patent B2
US 12,256,096 · App. 18/623,198 · Granted Mar 18, 2025

Global motion estimation using road and ground object labels for geometry-based point cloud compression

Inventors: Luong Pham Van (San Diego, CA); Adarsh Krishnan Ramasubramonian (Irvine, CA); Bappaditya Ray (San Diego, CA); Geert Van der Auwera (Del Mar, CA); Marta Karczewicz (San Diego, CA)
Assignee: QUALCOMM Incorporated
H04N19/527H04N19/124H04N19/139H04N19/177H04N19/597
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,256,096
App. No.
18/623,198
Granted
Mar 18, 2025
Kind
B2
Abstract

A device to code a point cloud data that includes a memory configured to store data representing points of a point cloud, and one or more processors implemented in circuitry and configured to: determine height values of points in a point cloud; code a data structure including data that represents a top threshold and a bottom threshold; classify points having height values between the top threshold and the bottom threshold into the set of ground points; classify points having height values above the top threshold or below the bottom threshold into the set of object points. The one or more processors code the ground points and the object points according to the classifications. The one or more processors code a geometry data unit header that includes data that overrides or refines the data of the data structure for the at least one of the top threshold or the bottom threshold.

Claims (58)

1. A device for decoding point cloud data, the device comprising:

a memory configured to store data representing points of a point cloud; and

a processing system comprising one or more processors implemented in circuitry, the processing system configured to:

determine height values of points in a point cloud;

decode a data structure including data representing a top threshold and a bottom threshold;

classify points having height values between the top threshold and the bottom threshold into the set of ground points;

classify points having height values above the top threshold or below the bottom threshold into the set of object points;

decode the ground points and the object points according to the classifications; and

decode a geometry data unit header (GDH) that includes data that overrides or refines the data of the data structure for the at least one of (i) the top threshold or (ii) the bottom threshold.

2. The device of claim 1 , wherein to decode the object points, the processing system is configured to:

derive a set of global motion information for the object points; and

predict the object points using the set of global motion information.

3. The device of claim 2 , wherein to derive the set of global motion information, the processing system is configured to derive a rotation matrix and a translation vector, and wherein to decode the object points, the processing system is configured to apply the rotation matrix and the translation vector to reference points of a reference frame.

4. The device of claim 3 , wherein to decode the object points, the processing system is further configured to:

determine local node motion vectors of nodes of a prediction tree, the nodes including respective sets of reference points of the reference frame; and

apply the local node motion vectors to the nodes.

5. The device of claim 1 , wherein the processing system is configured to derive the set of global motion information only for the object points.

6. The device of claim 1 , wherein the set of global motion information comprises a first set of global motion information, and wherein to decode the ground points, the processing system is configured to:

derive a second set of global motion information for the ground points; and

predict the ground points using the second set of global motion information.

7. The device of claim 6 , wherein the processing system is configured to derive the second set of global motion information only for the ground points.

8. A device for encoding point cloud data, the device comprising:

a memory configured to store data representing points of a point cloud; and

a processing system comprising one or more processors implemented in circuitry, the processing system configured to:

determine height values of points in a point cloud;

encode a data structure including data representing a top threshold and a bottom threshold;

classify points having height values between the top threshold and the bottom threshold into the set of ground points;

classify points having height values above the top threshold or below the bottom threshold into the set of object points; and

encode the ground points and the object points according to the classifications,

encode a geometry data unit header (GDH) that includes data that overrides or refines the data of the data structure for the at least one of (i) the top threshold or (ii) the bottom threshold.

9. The device of claim 8 , wherein to encode the object points, the processing system is configured to:

derive a set of global motion information for the object points; and

predict the object points using the set of global motion information.

10. The device of claim 9 , wherein to derive the set of global motion information, the processing system is configured to derive a rotation matrix and a translation vector, and wherein to encode the object points, the processing system is configured to apply the rotation matrix and the translation vector to reference points of a reference frame.

11. The device of claim 10 , wherein to encode the object points, the processing system is further configured to:

determine local node motion vectors of nodes of a prediction tree, the nodes including respective sets of reference points of the reference frame; and

apply the local node motion vectors to the nodes.

12. The device of claim 8 , wherein the processing system is configured to derive the set of global motion information only for the object points.

13. The device of claim 8 , wherein the set of global motion information comprises a first set of global motion information, and wherein to encode the ground points, the processing system is configured to:

derive a second set of global motion information for the ground points; and

predict the ground points using the second set of global motion information.

14. The device of claim 13 , wherein the processing system is configured to derive the second set of global motion information only for the ground points.

15. A method of coding point cloud data, the method comprising:

determining height values of points in a point cloud;

coding a data structure including data representing a top threshold and a bottom threshold;

classifying points having height values between the top threshold and the bottom threshold into the set of ground points;

classifying points having height values above the top threshold or below the bottom threshold into the set of object points;

coding the ground points and the object points according to the classifications; and

coding a geometry data unit header (GDH) that includes data that overrides or refines the data of the data structure for the at least one of (i) the top threshold or (ii) the bottom threshold.

16. The method of claim 15 , wherein coding the object points comprises:

deriving a set of global motion information for the object points; and

predicting the object points using the set of global motion information.

17. The method of claim 16 , wherein deriving the set of global motion information comprises deriving the set of global motion information only for the object points.

18. The method of claim 16 , wherein the set of global motion information comprises a first set of global motion information, and wherein coding the ground points comprises:

deriving a second set of global motion information for the ground points; and

predicting the ground points using the second set of global motion information.

19. The method of claim 15 , wherein the coding includes encoding.

20. The method of claim 15 , wherein the coding includes decoding.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 3, 2024
From: PHAM VAN, LUONG; RAMASUBRAMONIAN, ADARSH KRISHNAN; RAY, BAPPADITYA; VAN DER AUWERA, GEERT; KARCZEWICZ, MARTA
To: QUALCOMM INCORPORATED
Reel/Frame 067596/0244 →
Continuity (4)
Continuation 17558362 · Dec 21, 2021
Provisional Application 63171945 · Apr 7, 2021
Provisional Application 63131637 · Dec 29, 2020
Related Publication 20240251097A1 · Jul 25, 2024
References Cited (50)
US 6097698A · Yang · 2000 [cited by examiner]
US 6239805B1 · Deering · 2001 [cited by examiner]
US 20030108099A1 · Nagumo · 2003 [cited by examiner]
US 20090079876A1 · Takeshima · 2009 [cited by examiner]
US 20100150242A1 · Ozawa · 2010 [cited by examiner]
US 20110080762A1 · Nikolov · 2011 [cited by applicant]
US 20130083162A1 · Wang · 2013 [cited by examiner]
US 20140192050A1 · Qiu et al. · 2014 [cited by applicant]
US 20150356357A1 · McMANUS et al. · 2015 [cited by applicant]
US 20160256127A1 · Lee · 2016 [cited by examiner]
US 20170236288A1 · Sundaresan · 2017 [cited by examiner]
US 20170287143A1 · Shimada · 2017 [cited by examiner]
US 20170323423A1 · Lin · 2017 [cited by examiner]
US 20180007155A1 · Saito · 2018 [cited by examiner]
US 20180232947A1 · Nehmadi et al. · 2018 [cited by applicant]
US 20190138813A1 · Pereira · 2019 [cited by examiner]
US 20190156507A1 · Zeng · 2019 [cited by examiner]
US 20190204076A1 · Nishi et al. · 2019 [cited by applicant]
US 20190242711A1 · Ingersoll et al. · 2019 [cited by applicant]
US 20190324471A1 · Kim · 2019 [cited by examiner]
US 20190325089A1 · Golparvar-Fard et al. · 2019 [cited by applicant]
US 20200043121A1 · Boyce · 2020 [cited by examiner]
US 20200050878A1 · Badr · 2020 [cited by examiner]
US 20200082611A1 · Haramaty et al. · 2020 [cited by applicant]
US 20200219286A1 · Sinharoy · 2020 [cited by examiner]
US 20200280710A1 · Vosoughi et al. · 2020 [cited by applicant]
US 20200396479A1 · Furht · 2020 [cited by examiner]
US 20200396501A1 · Lapicque · 2020 [cited by examiner]
US 20210000006A1 · Ellaboudy et al. · 2021 [cited by applicant]
US 20210051320A1 · Tourapis · 2021 [cited by examiner]
US 20210092432A1 · Rusanovskyy · 2021 [cited by examiner]
US 20210116568A1 · Kanza et al. · 2021 [cited by applicant]
US 20210192798A1 · Lasserre et al. · 2021 [cited by applicant]
US 20220210466A1 · Pham Van et al. · 2022 [cited by applicant]
US 20220215596A1 · Van Der Auwera et al. · 2022 [cited by applicant]
CN 110782465B · 2020 [cited by applicant]
EP 3633621A1 · 2020 [cited by applicant]
3DG: “G-PCC Codec Description v8”, 131. MPEG Meeting, Jun. 29, 2020-Jul. 3, 2020, Online, (Motion Picture Expert Group or ISO/IEC JTC1/SC29/WG11), Coding of Moving Pictures and Audio, Convenorship: JISC (Japan), No. N19… [cited by applicant]
International Preliminary Report on Patentability—PCT/US2021/064869—The International Bureau of WIPO—Geneva, Switzerland—Jul. 13, 2023. [cited by applicant]
International Search Report and Written Opinion—PCT/US2021/064869—ISA/EPO—Sep. 9, 2022. [cited by applicant]
International Search Report and Written Opinion—PCT/US2021/065343—ISA/EPO—Apr. 21, 2022. [cited by applicant]
ITU-T H.265: “Series H: Audiovisual and Multimedia Systems Infrastructure of Audiovisual Services—Coding of Moving Video”, High Efficiency Video Coding, The International Telecommunication Union, Jun. 2019, 696 Pages. [cited by applicant]
Kim J., et al., “[G-PCC] [EM Related] [New Proposal] Road and Objects Segmentation for Global Motion Prediction”, 132. MPEG Meeting, Oct. 12, 2020-Oct. 16, 2020, Online, (Motion Picture Expert Group or ISO/IEC JTC1/SC29… [cited by applicant]
Lasserre (Blackberry) S., et al., “[PCC] Global Motion Compensation for Point Cloud Compression in TMC3”, 124. MPEG, Meeting, Oct. 8, 2018-Oct. 12, 2018, Macao, (Motion Picture Expert Group or ISO/IEC JTC1/SC29/WG11), N… [cited by applicant]
Lasserre (Blackberry) S., et al., “[PCC] On Motion Compensation for Geometry Coding in TM3”, 122. MPEG Meeting, Apr. 16, 2018-Apr. 20, 2018, San Diego, (Motion Picture Expert Group or ISO/IEC JTC1/SC29/WG11), No. m42521… [cited by applicant]
Lasserre S., “Exploratory Model for Inter-Prediction in G-PCC”, 124. MPEG Meeting, Oct. 8, 2018-Oct. 12, 2018, MACAO, (Motion Picture Expert Group or ISO/IEC JTC1/SC29/WG11), No. n18096, Oct. 31, 2018, XP030193470, 4 Pa… [cited by applicant]
“NumPy”, numpy.org, 2021, pp. 1-3. [cited by applicant]
Partial International Search Report—PCT/US2021/064869—ISA/EPO—May 18, 2022. [cited by applicant]
U.S. Appl. No. 17/495,428, filed Oct. 6, 2021, by Cao et al. [cited by applicant]
Yin H., et al., “3D LiDAR Map Compression for Efficient Localization on Resource Constrained Vehicles”, IEEE Transactions on Intelligent Transportation Systems, 2020, pp. 1-16. [cited by applicant]
Cited By (2)
US 12,671,837 US 12,739,395