IP Library Granted Patent US 11,328,401
Granted Patent B2
US 11,328,401 · App. 16/517,782 · Granted May 10, 2022

Stationary object detecting method, apparatus and electronic device

Inventors: Weixin Lu (Beijing, CN); Shiyu Song (Sunnyvale, CA); Fangfang Dong (Beijing, CN); Shengpan Xu (Beijing, CN)
G06T7/0002G06F17/18G06K9/6218G06K9/6276G06T7/10G06T2207/10028
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,328,401
App. No.
16/517,782
Granted
May 10, 2022
Kind
B2
Abstract

A stationary object detecting method, a stationary object apparatus, and an electronic device are disclosed in embodiments of the present disclosure, the method includes: obtaining point cloud data of a scene and feature information of each of data points in the point cloud data; performing a triangulation network connection on each of the data points in the point cloud data to generate a triangular network model, and taking a picture of the triangular network model using a preset camera to obtain an image; obtaining a first data point corresponding to each pixel point in the image, and obtaining a feature map of the point cloud data according to feature information of each first data point; inputting the feature map into a classification model to obtain data points corresponding to the stationary object in the point cloud data.

Claims (66)

1. A stationary object detecting method, comprising:

obtaining point cloud data of a scene, wherein a background, a moving ground object in a moving state and a stationary object temporarily in a stationary state are comprised in the scene, and the stationary object comprises a vehicle in the stationary state;

performing a pre-processing on the point cloud data to remove the moving ground object in the scene and generate evenly distributed point cloud data that comprises data corresponding to the background and the stationary object;

obtaining feature information of each of data points in the pre-processed point cloud data;

performing a triangulation network connection on the data points in the pre-processed point cloud data to generate a triangular network model, and taking a picture of the triangular network model using a preset camera to obtain an image, wherein each pixel point in the image corresponds to one of a plurality of triangles in the triangular network model;

obtaining a first data point corresponding to each pixel point in the image, and obtaining a feature map of the pre-processed point cloud data according to feature information of each first data point, wherein the first data point is a data point on a vertex of the triangle corresponding to the pixel point; and

inputting the feature map into a classification model to obtain data points corresponding to the stationary object in the pre-processed point cloud data;

wherein the inputting the feature map into the classification model to obtain the data points corresponding to the stationary object in the pre-processed point cloud data comprises:

inputting the feature map into the classification model to obtain a category of each pixel point in the image, wherein the category comprises a background or stationary object;

determining, according to the category of each pixel point, whether a category of a data point corresponding to each pixel point is a background or stationary object;

clustering each of the data points in the pre-processed point cloud data according to categories of the data points and a distance between each two of the data points to obtain at least one data cluster; and

determining the data points corresponding to the stationary object according to a shape of each data cluster and a number of data points comprised in each data cluster.

2. The method according to claim 1 , wherein the feature information of the data points comprises at least one of intensity information, a planar probability, a scattered probability, depth information, an optimal domain, and a vertical component of a normal vector of the optimal domain of the data points.

3. The method according to claim 2 , wherein the feature map comprises a first feature map and a second feature map, and the obtaining a feature map of the pre-processed point cloud data according to feature information of each first data point comprises:

generating the first feature map of the pre-processed point cloud data according to intensity information, a planar probability, and a scattered probability of each first data point;

generating the second feature map of the pre-processed point cloud data according to an optimal domain, depth information, and a vertical component of a normal vector of the optimal domain of each first data point; and

the inputting the feature map into a classification model to obtain data points corresponding to the stationary object in the pre-processed point cloud data comprises:

inputting the first feature map and the second feature map into the classification model to obtain the stationary object in the pre-processed point cloud data.

4. The method according to claim 1 , wherein the classification model

is a SnapNet classification model, wherein the SnapNet classification model is a two-class network model for distinguishing between a background and a stationary object.

5. The method according to claim 1 , wherein after the determining the data points corresponding to the stationary object according to a shape of each data cluster and a number of data points comprised in each data cluster, the method further comprises:

performing an oriented bounding box filtering on the data points corresponding to the stationary object to enclose the data points corresponding to the stationary object in the oriented bounding box.

6. The method according to claim 1 , wherein before the taking a picture of the triangular network model using a preset camera, the method further comprises:

generating a camera parameter according to a preset constraint, wherein the constraint comprises that: a pitch angle of a camera is greater than or equal to a preset angle, and a proportion of effective pixels in a generated image is higher than a preset value; and

generating the preset camera according to the camera parameter.

7. The method according to claim 2 , wherein the feature information comprises the optimal domain, and the obtaining feature information of each of data points in the pre-processed point cloud data comprises:

for each data point, obtaining all domains of the each data point, and a linear probability, a planar probability and a scattered probability of each of the domains;

determining an amount of domain information of each of the domains according to the linear probability, the planar probability, and the scattered probability of each of the domains; and

taking a domain with a smallest amount of domain information as the optimal domain of the each data point, and taking a linear probability, a planar probability and a scattered probability of the optimal domain as the linear probability, the planar probability and the scattered probability of the each data point, respectively.

8. A stationary object detecting apparatus, comprising a memory and a processor, wherein the memory is configured to store a computer program; and

the processor, when executing the computer program, is configured to:

obtain point cloud data of a scene, wherein a background, a moving ground object in a moving state and a stationary object temporarily in a stationary state are comprised in the scene, and the stationary object comprises a vehicle in the stationary state;

perform a pre-processing on the point cloud data to remove the moving ground object in the scene and generate evenly distributed point cloud data that comprises data corresponding to the background and the stationary object;

obtain feature information of each of data points in the pre-processed point cloud data

establish a triangular network model based on the data points in the pre-processed point cloud data;

take a picture of the triangular network model using a preset camera to obtain an image, wherein each pixel point in the image corresponds to a triangle in the triangular network model;

obtain a first data point corresponding to each pixel point in the image, and obtain a feature map of the pre-processed point cloud data according to feature information of each first data point, wherein the first data point is a data point on a vertex of a triangle corresponding to each pixel point;

input the feature map into a classification model to obtain a category of each pixel point in the image, wherein the category comprises a background or stationary object;

determine, according to the category of each pixel point, whether a category of a data point corresponding to each pixel point is a background or stationary object;

cluster each of the data points in the pre-processed point cloud data according to categories of the data points and a distance between each two of the data points to obtain at least one data cluster;

determine the data points corresponding to the stationary object according to a shape of each data cluster and a number of data points comprised in each data cluster.

9. The apparatus according to claim 8 , wherein the feature information of the data points comprises at least one of intensity information a planar probability, a scattered probability, depth information, an optimal domain, and a vertical component of a normal vector of the optimal domain of the data points.

10. The apparatus according to claim 9 , wherein the feature map comprises a first feature map and a second feature map, and the processor is configured to:

generate the first feature map of the pre-processed point cloud data according to intensity information, a planar probability, and a scattered probability of each first data point; and generate the second feature map of the pre-processed point cloud data according to an optimal domain, depth information, and a vertical component of a normal vector of the optimal domain of each first data point; and

input the first feature map and the second feature map into the classification model to obtain the stationary object in the pre-processed point cloud data.

11. The apparatus according to claim 8 , wherein,

the processor is configured to input the feature map into a SnapNet classification model to obtain a category of each pixel point in the image, wherein the SnapNet classification model is a two-class network model for distinguishing between a background and a stationary object.

12. The apparatus according to claim 8 , wherein the processor is further configured to:

perform an oriented bounding box filtering on the data points corresponding to the stationary object to enclose the data points corresponding to the stationary object in the oriented bounding box.

13. The apparatus according to claim 8 , wherein the processor is further configured to:

generate a camera parameter according to a preset constraint, wherein the constraint comprises that: a pitch angle of a camera is greater than or equal to a preset angle, and a proportion of effective pixels in a generated image is higher than a preset value; and

generate the preset camera according to the camera parameter.

14. The apparatus according to claim 9 , wherein the feature information comprises an optimal domain, and the processor is configured to: for each data point, obtain all domains of the each data point, and a linear probability, a planar probability and a scattered probability of each of the domains; determine an amount of domain information of each of the domains according to the linear probability, the planar probability, and the scattered probability of each of the domains; take a domain with a smallest amount of domain information as the optimal domain of the each data point, and take a linear probability, a planar probability and a scattered probability of the optimal domain as the linear probability, the planar probability and the scattered probability of the each data point, respectively.

15. A non-volatile computer storage medium, wherein the storage medium stores a computer program, wherein the computer program, when being executed, implements the following:

obtaining point cloud data of a scene, wherein a background, a moving ground object in a moving state and a stationary object temporarily in a stationary state are comprised in the scene, and the stationary object comprises a vehicle in the stationary state;

performing a pre-processing on the point cloud data to remove the moving ground object in the scene and generate evenly distributed point cloud data that comprises data corresponding to the background and the stationary object;

obtaining feature information of each of data points in the pre-processed point cloud data;

establishing a triangular network model based on the data points in the pre-processed point cloud data, and taking a picture of the triangular network model using a preset camera to obtain an image, wherein each pixel point in the image corresponds to a triangle in the triangular network model;

obtaining a first data point corresponding to each pixel point in the image, and obtaining a feature map of the pre-processed point cloud data according to feature information of each first data point, wherein the first data point is a data point on a vertex of a triangle corresponding to each pixel point; and

inputting the feature map into a classification model to obtain data points corresponding to the stationary object in the pre-processed point cloud data;

wherein the inputting the feature map into a classification model to obtain data points corresponding to the stationary object in the pre-processed point cloud data comprises:

inputting the feature map into the classification model to obtain a category of each pixel point in the image, wherein the category comprises a background or stationary object;

determining, according to the category of each pixel point, whether a category of a data point corresponding to each pixel point is a background or stationary object;

clustering each of the data points in the pre-processed point cloud data according to categories of the data points and a distance between each two of the data points to obtain at least one data cluster; and

determining the data points corresponding to the stationary object according to a shape of each data cluster and a number of data points comprised in each data cluster.

16. The non-volatile computer storage medium according to claim 15 , wherein the feature information of the data points comprises at least one of intensity information, a planar probability, a scattered probability, depth information, an optimal domain, and a vertical component of a normal vector of the optimal domain of the data points.

Assignments (6)
CORRECTIVE ASSIGNMENT TO CORRECT THE APPLICANT NAME PREVIOUSLY RECORDED AT REEL: 057933 FRAME: 0812. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Dec 28, 2021
From: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
To: APOLLO INTELLIGENT DRIVING TECHNOLOGY (BEIJING) CO., LTD.
Reel/Frame 058594/0836 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 13, 2021
From: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
To: APOLLO INTELLIGENT DRIVING (BEIJING) TECHNOLOGY CO., LTD.
Reel/Frame 057933/0812 →
CORRECTIVE ASSIGNMENT TO CORRECT THE RECEIVING PARTY DATA PREVIOUSLY RECORDED AT REEL: 055533 FRAME: 0127. ASSIGNOR(S) HEREBY CONFIRMS THE EMPLOYMENT AGREEMENT . Recorded Mar 12, 2021
From: XU, SHENGPAN
To: BAIDU.COM TIMES TECHNOLOGY (BEIJING) CO., LTD.
Reel/Frame 055575/0250 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 10, 2021
From: BAIDU.COM TIMES TECHNOLOGY (BEIJING) CO. LTD.
To: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
Reel/Frame 055543/0223 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 9, 2021
From: LU, WEIXIN; SONG, SHIYU; DONG, FANGFANG
To: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
Reel/Frame 055532/0383 →
EMPLOYMENT AGREEMENT WITH ENGLISH TRANSLATION Recorded Mar 9, 2021
From: XU, SHENGPAN
To: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
Reel/Frame 055533/0127 →
Priority Claims (1)
CN 201810880407.6 · Aug 3, 2018 · national
Continuity (1)
Related Publication 20190340746A1 · Nov 7, 2019
Cited By (2)
US 12,394,063 US 12,555,245