IP Library Granted Patent US 11,875,424
Granted Patent B2
US 11,875,424 · App. 17/735,888 · Granted Jan 16, 2024

Point cloud data processing method and device, computer device, and storage medium

Inventors: Hui Huang (Shenzhen, CN); Pengdi Huang (Shenzhen, CN)
Assignee: Shenzhen University
G06T9/002G06V10/7715G06V20/58G06V20/64G06V20/70
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,875,424
App. No.
17/735,888
Granted
Jan 16, 2024
Kind
B2
Abstract

A point cloud data processing method and device, a computer device and a storage medium are provided. The method includes: acquiring point cloud data, and constructing a corresponding neighboring point set for each of data points in the point cloud data; calculating Hausdorff distances between the neighboring point set and a pre-constructed kernel point cloud to obtain a distance matrix; calculating a convolution of the neighboring point set with the distance matrix and a network weight matrix in a Hausdorff convolution layer in an encoder, to obtain high-dimensional point cloud features, the encoder and a decoder being two parts in a deep learning network; and reducing feature dimension of the high-dimensional point cloud features through the decoder, so that a classifier performs semantic classification on the point cloud data according to object point cloud features obtained by the dimension reduction.

Claims (48)

1. A point cloud data processing method, comprising:

acquiring point cloud data, and constructing a corresponding neighboring point set for each of data points in the point cloud data;

calculating Hausdorff distances between the neighboring point set and a pre-constructed kernel point cloud to obtain a distance matrix;

calculating a convolution of the neighboring point set with the distance matrix and a network weight matrix in a Hausdorff convolution layer in an encoder to obtain high-dimensional point cloud features; the encoder and a decoder being two parts in a deep learning network; and

reducing feature dimension of the high-dimensional point cloud features through the decoder, so that a classifier performs semantic classification on the point cloud data according to object point cloud features obtained by the dimension reduction.

2. The point cloud data processing method according to claim 1 , wherein the number of Hausdorff convolutional layers in the encoder is not less than two; and the calculating a convolution of the neighboring point set with the distance matrix and the network weight matrix in the Hausdorff convolution layer in the encoder to obtain the high-dimensional point cloud features comprises:

in a process of calculating convolution in a first Hausdorff convolution layer, convolving the neighboring point set, which is used as input features, with the distance matrix and the network weight matrix, to obtain output features; and

in a non-first Hausdorff convolutional layer, convolving output features of a previous Hausdorff convolutional layer of a current non-first Hausdorff convolutional layer, which are used as input features of the current non-first Hausdorff convolutional layer, with the distance matrix and the network weight matrix, to obtain output features, and if the current non-first Hausdorff convolutional layer is a last Hausdorff convolutional layer, using output features of the last Hausdorff convolution layer as the high-dimensional point cloud features.

3. The point cloud data processing method according to claim 1 , wherein the constructing the corresponding neighboring point set for each of data points in the point cloud data comprises:

selecting a query point from the data points of the point cloud data;

extracting, from the data points of the point cloud data, object data points whose distances from the query point each are smaller than a query radius; and

constructing a neighboring point set corresponding to the query point according to the object data points.

4. The point cloud data processing method according to claim 1 , wherein the Hausdorff distances comprise a first shortest distance and a second shortest distance; and the calculating the Hausdorff distances between the neighboring point set and the pre-constructed kernel point cloud to obtain the distance matrix comprises:

calculating first shortest distances between the data points in the neighboring point set and the kernel point cloud, respectively, and forming the first shortest distances into a first shortest distance set;

calculating second shortest distances between data points in the kernel point cloud and the neighboring point set, respectively, and forming the second shortest distances into a second shortest distance set; and

calculating the distance matrix according to the first shortest distance set and the second shortest distance set.

5. The point cloud data processing method according to claim 1 , wherein, prior to the calculating the Hausdorff distances between the neighboring point set and the pre-constructed kernel point cloud, the method further comprises:

creating a spherical region, and constructing a three-dimensional parametric model in the spherical region; and

sampling the three-dimensional parametric model through a farthest point algorithm, and constructing the kernel point cloud.

6. The point cloud data processing method according to claim 1 , wherein:

after the calculating the Hausdorff distances between the neighboring point set and the pre-constructed kernel point cloud to obtain the distance matrix, the method further comprises:

calculating ratios of non-zero elements in the distance matrix to a query radius; and

calculating and obtaining a normalized distance matrix according to the ratios; and

the calculating the convolution of the neighboring point set with the distance matrix and the network weight matrix in the Hausdorff convolution layer in the encoder comprises:

convolving the neighboring point set, with the normalized distance matrix and the network weight matrix in the Hausdorff convolution layer in the encoder.

7. The point cloud data processing method according to claim 1 , wherein:

the Hausdorff convolution layer in the encoder comprises multi-kernel Hausdorff convolution sublayers;

different multi-kernel Hausdorff convolution sublayers correspond to different distance matrixes; and

the calculating the convolution of the neighboring point set with the distance matrix and the network weight matrix in the Hausdorff convolution layer in the encoder to obtain the high-dimensional point cloud features comprises:

in each of the multi-kernel Hausdorff convolution sublayers, convolving the neighboring point set with the network weight matrix and the distance matrix corresponding to each of the multi-kernel Hausdorff convolution sublayers to obtain point cloud features of each of the multi-kernel Hausdorff convolution sublayers; and

merging the point cloud features of each of the multi-kernel Hausdorff convolution sublayers to obtain the high-dimensional point cloud features.

8. The point cloud data processing method according to claim 1 , wherein each Hausdorff convolutional layer in the encoder comprises a plurality of Hausdorff convolution modules, and different Hausdorff convolution modules have different query scales.

9. The point cloud data processing method according to claim 1 , wherein the point cloud data is a three-dimensional point cloud data.

10. The point cloud data processing method according to claim 1 , wherein the point cloud data is obtained by scanning objects in a road environment by a lidar on a driverless vehicle.

11. The point cloud data processing method according to claim 1 , wherein the point cloud data is obtained by scanning objects in a space environment by a depth camera on an artificial intelligence robot.

12. The point cloud data processing method according to claim 2 , wherein a query radius of a current Hausdorff convolutional layer is twice that of a previous Hausdorff convolutional layer.

13. The point cloud data processing method according to claim 2 , wherein five Hausdorff convolutional layers are configured in the encoder.

14. The point cloud data processing method according to claim 6 , wherein elements in the normalized distance matrix are 1−M min (i, j)/R, wherein, M min denotes the distance matrix, R denotes the query radius, M min (i, j)/R denotes ratios of the non-zero elements in the distance matrix to the query radius.

15. The point cloud data processing method according to claim 7 , wherein the merging the point cloud features of each of the multi-kernel Hausdorff convolution sublayers comprising adding or weighted adding the point cloud features.

16. The point cloud data processing method according to claim 7 , wherein the merging the point cloud features of each of the multi-kernel Hausdorff convolution sublayers to obtain the high-dimensional point cloud features comprises: splicing convolution response vectors output by the multi-kernel Hausdorff convolution sublayers to merge the convolution response vectors to obtain the high-dimensional point cloud features.

17. The point cloud data processing method according to claim 7 , wherein the merging the point cloud features of each of the multi-kernel Hausdorff convolution sublayers to obtain the high-dimensional point cloud features comprises: mapping convolution response vectors, output by the multiple multi-kernel Hausdorff convolution sublayers, onto one-dimensional data through a multi-layer perceptron to merge the convolution response vectors to obtain the high-dimensional point cloud features.

18. A computer device, comprising a memory storing a computer program, and a processor, wherein, the processor, when executing the computer program, performs steps of the method according to claim 1 .

19. A non-transitory computer-readable storage medium, on which a computer program is stored, wherein, the computer program, when executed by a processor, causes the processor to perform steps of the method according to claim 1 .

20. A point cloud data processing device, comprising:

an acquiring module, configured to acquire point cloud data, and construct a corresponding neighboring point set for each of data points in the point cloud data;

a distance calculation module, configured to calculate Hausdorff distances between the neighboring point set and a pre-constructed kernel point cloud to obtain a distance matrix;

a convolution calculation module, configured to calculate a convolution of the neighboring point set with the distance matrix and a network weight matrix in a Hausdorff convolution layer in an encoder to obtain high-dimensional point cloud features; the encoder and a decoder being two parts in a deep learning network; and

a feature dimension reduction module, configured to reduce feature dimension of the high-dimensional point cloud features through the decoder, so that a classifier performs semantic classification on the point cloud data according to object point cloud features obtained by the dimension reduction.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 5, 2022
From: HUANG, HUI; HUANG, PENGDI
To: SHENZHEN UNIVERSITY
Reel/Frame 059832/0553 →
Priority Claims (1)
CN 202110274439.3 · Mar 15, 2021 · national
Continuity (2)
Continuation PCTCN2021082250 · Mar 23, 2021
Related Publication 20220292728A1 · Sep 15, 2022