IP Library Granted Patent US 11,468,689
Granted Patent B2
US 11,468,689 · App. 17/017,508 · Granted Oct 11, 2022

Method and apparatus for detecting obstacle, electronic device and storage medium

Inventors: Zhi Wang (Beijing, CN); Yong Pan (Beijing, CN); Qinghui Zhao (Beijing, CN)
Assignee: APOLLO INTELLIGENT DRIVING TECHNOLOGY (BEIJING) CO., LTD.
G06V20/58G01S7/412G01S13/867G01S13/931G06T7/50G06T7/70G06V10/25G06V20/588G06T2207/30256G06T2207/30261
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,468,689
App. No.
17/017,508
Granted
Oct 11, 2022
Kind
B2
Abstract

A method and apparatus for detecting an obstacle, an electronic device, and a storage medium. A specific implementation of the method includes: detecting, by a millimeter-wave radar, position points of candidate obstacles in front of a vehicle; detecting, by a camera, a left road boundary line and a right road boundary line of a road on which a vehicle is located; separating the position points of the candidate obstacles according to the left road boundary line and the right road boundary line of the road on which the vehicle is located, and extracting position points between the left road boundary line and the right road boundary line; projecting the position points between the left road boundary line and the right road boundary line onto an image; and detecting, based on projection points of the position points on the image, a target obstacle in front of the vehicle.

Claims (59)

1. A method for detecting an obstacle, comprising:

detecting, by a millimeter-wave radar, position points of candidate obstacles in front of a vehicle;

detecting, by a camera, a left road boundary line and a right road boundary line of a road on which a vehicle is located;

separating the position points of the candidate obstacles according to the left road boundary line and the right road boundary line of the road on which the vehicle is located, and extracting position points between the left road boundary line and the right road boundary line;

projecting the position points between the left road boundary line and the right road boundary line onto an image;

acquiring, on the image, regions of interest (ROIs) corresponding to the projection points;

calculating detection results of the ROIs corresponding to the projection points through a depth convolution network, wherein a detection result comprises: a detection result indicting existence of an obstacle or a detection result indicting non-existence of an obstacle;

extracting from the image, based on pre-determined position points falsely-detected by the radar, projection points of the falsely-detected position points on the image;

filtering out, in the detection results of the ROIs corresponding to the projection points outputted by the depth convolution network, the projection points of the falsely-detected position points and detection results of ROIs corresponding to the projection points of the falsely-detected position points;

determining the obstacle in front of the vehicle based on detection results of ROIs corresponding to projection points after the filtration; and

detecting the target obstacle in front of the vehicle according to the detection results of the ROIs corresponding to the projection points.

2. The method according to claim 1 , wherein the calculating the detection results of the ROIs corresponding to the projection points through the depth convolution network comprises:

inputting ROIs corresponding to the projection points into the depth convolution network;

acquiring, in the ROIs corresponding to the projection points, perception results of the position points;

calculating the detection results of the ROIs corresponding to the projection points based on the perception results of the position points; and

outputting, by the depth convolution network, the detection results of the ROIs corresponding to the projection points.

3. The method according to claim 1 , wherein the detecting, based on the projection points of the position points on the image, the target obstacle in front of the vehicle comprises:

detecting a type of the target obstacle according to projection points of the position points on the image; and

calculating a distance between the target obstacle and the vehicle based on the projection points of the position points on the image and predetermined position points occupied by the vehicle.

4. An apparatus for detecting an obstacle, comprising:

at least one processor; and

a memory storing instructions, the instructions, when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising:

detecting, by a millimeter-wave radar, position points of candidate obstacles in front of a vehicle;

detecting, by a camera, a left road boundary line and a right road boundary line of a road on which a vehicle is located;

separating the position points of the candidate obstacles according to the left road boundary line and the right road boundary line of the road on which the vehicle is located, and extracting position points between the left road boundary line and the right road boundary line; and

projecting the position points between the left road boundary line and the right road boundary line onto an image;

acquiring, on the image, regions of interest (ROIs) corresponding to the projection points;

calculating detection results of the ROIs corresponding to the projection points through a depth convolution network, wherein a detection result comprises: a detection result indicting existence of an obstacle or a detection result indicting non-existence of an obstacle;

extracting from the image, based on pre-determined position points falsely-detected by the radar, projection points of the falsely-detected position points on the image;

filtering out, in the detection results of the ROIs corresponding to the projection points outputted by the depth convolution network, the projection points of the falsely-detected position points and detection results of ROIs corresponding to the projection points of the falsely-detected position points;

determining the obstacle in front of the vehicle based on detection results of ROIs corresponding to projection points after the filtration; and

detecting the target obstacle in front of the vehicle according to the detection results of the ROIs corresponding to the projection points.

5. The apparatus according to claim 4 , wherein the calculating the detection results of the ROIs corresponding to the projection points through the depth convolution network comprises:

inputting ROIs corresponding to the projection points into the depth convolution network;

acquiring, in the ROIs corresponding to the projection points, perception results of the position points;

calculating the detection results of the ROIs corresponding to the projection points based on the perception results of the position points; and

outputting, by the depth convolution network, the detection results of the ROIs corresponding to the projection points.

6. The apparatus according to claim 4 , wherein the detecting, based on the projection points of the position points on the image, the target obstacle in front of the vehicle comprises:

detecting a type of the target obstacle according to projection points of the position points on the image; and

calculating a distance between the target obstacle and the vehicle based on the projection points of the position points on the image and predetermined position points occupied by the vehicle.

7. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions, when executed by a processor, cause the processor to perform operations, the operations comprising:

detecting, by a millimeter-wave radar, position points of candidate obstacles in front of a vehicle;

detecting, by a camera, a left road boundary line and a right road boundary line of a road on which a vehicle is located;

separating the position points of the candidate obstacles according to the left road boundary line and the right road boundary line of the road on which the vehicle is located, and extracting position points between the left road boundary line and the right road boundary line;

projecting the position points between the left road boundary line and the right road boundary line onto an image;

acquiring, on the image, regions of interest (ROIs) corresponding to the projection points;

calculating detection results of the ROIs corresponding to the projection points through a depth convolution network, wherein a detection result comprises: a detection result indicting existence of an obstacle or a detection result indicting non-existence of an obstacle;

extracting from the image, based on pre-determined position points falsely-detected by the radar, projection points of the falsely-detected position points on the image;

filtering out, in the detection results of the ROIs corresponding to the projection points outputted by the depth convolution network, the projection points of the falsely-detected position points and detection results of ROIs corresponding to the projection points of the falsely-detected position points;

determining the obstacle in front of the vehicle based on detection results of ROIs corresponding to projection points after the filtration; and

detecting the target obstacle in front of the vehicle according to the detection results of the ROIs corresponding to the projection points.

8. The medium according to claim 7 , wherein the calculating the detection results of the ROIs corresponding to the projection points through the depth convolution network comprises:

inputting ROIs corresponding to the projection points into the depth convolution network;

acquiring, in the ROIs corresponding to the projection points, perception results of the position points;

calculating the detection results of the ROIs corresponding to the projection points based on the perception results of the position points; and

outputting, by the depth convolution network, the detection results of the ROIs corresponding to the projection points.

9. The medium according to claim 7 , wherein the detecting, based on the projection points of the position points on the image, the target obstacle in front of the vehicle comprises:

detecting a type of the target obstacle according to projection points of the position points on the image; and

calculating a distance between the target obstacle and the vehicle based on the projection points of the position points on the image and predetermined position points occupied by the vehicle.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 13, 2021
From: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY CO., LTD.
To: APOLLO INTELLIGENT DRIVING TECHNOLOGY (BEIJING) CO., LTD.
Reel/Frame 058241/0248 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 14, 2020
From: WANG, ZHI; PAN, YONG; ZHAO, QINGHUI
To: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY CO., LTD.
Reel/Frame 053769/0652 →
Priority Claims (1)
CN 202010166864.6 · Mar 11, 2020 · national
Continuity (1)
Related Publication 20210287019A1 · Sep 16, 2021