IP Library Granted Patent US 11,126,875
Granted Patent B2
US 11,126,875 · App. 16/508,918 · Granted Sep 21, 2021

Method and device of multi-focal sensing of an obstacle and non-volatile computer-readable storage medium

Inventors: Han Gao (Beijing, CN); Guo Li (Beijing, CN); Xun Sun (Beijing, CN); Tian Xia (Beijing, CN)
Assignee: Baidu Online Network Technology (Beijing) Co., Ltd.
G06K9/00825G06K9/00805H04N5/23218H04N5/232122
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,126,875
App. No.
16/508,918
Granted
Sep 21, 2021
Kind
B2
Abstract

A method and device of multi-focal sensing of an obstacle. A method includes acquiring detection results of obstacles at multiple moments by utilizing a camera with long focus lens and a camera with short focus lens; performing a target tracking to the acquired detection results, to obtain at least two tracking sequences, wherein each tracking sequence includes detection results acquired at the multiple moments for a same obstacle; and matching two random tracking sequences of the at least two tracking sequences, and combining the two random tracking sequences into a combined tracking sequence, if the matching is successful.

Claims (35)

1. A method for multi-focal sensing of an obstacle, comprising:

acquiring detection results of obstacles at multiple moments by utilizing a camera with long focus lens and a camera with short focus lens;

performing a target tracking to the acquired detection results, to obtain at least two tracking sequences, wherein each tracking sequence comprises detection results acquired at the multiple moments for a same obstacle; and

matching two random tracking sequences of the at least two tracking sequences, and combining the two random tracking sequences into a combined tracking sequence, if the matching is successful.

2. The method according to claim 1 , wherein the matching two random tracking sequences of the at least two tracking sequences comprises:

selecting the two random tracking sequences from the at least two tracking sequences;

acquiring, for each moment, detection results from the selected two tracking sequences respectively, and projecting detection results acquired by the camera with the long focus lens onto an imaging plane acquired by the camera with the short focus lens, if the detection results from the selected two tracking sequences respectively are acquired by cameras with different focus lenses, and calculating an Intersection over Union (IoU) of the projected detection results and detection results acquired by the camera with the short focus lens;

determining an average of the IoUs calculated for the multiple moments; and

determining the matching of the selected two tracking sequences is successful if the average is greater than a preset threshold.

3. The method according to claim 1 , wherein the combining the two random tracking sequences into a combined tracking sequence comprises:

combining detection results of the two random tracking sequences to obtain a combined tracking sequence.

4. The method according to claim 1 , further comprising:

acquiring a new detection result, and determining whether the new detection result is correlated with the combined tracking sequence;

creating a new tracking sequence if the new detection result is not correlated with the combined tracking sequence, wherein the new tracking sequence comprises the new detection result; or

incorporating the new detection result into the combined tracking sequence if the new detection result is correlated with the combined tracking sequence.

5. A device of multi-focal sensing of an obstacle, comprising:

a camera with long focus lens for acquiring detection results of obstacles;

a camera with short focus lens for acquiring detection results of obstacles; and

a controller comprising a memory and a processor, wherein the memory has computer programs stored thereon, and the computer programs are executed by the processor to cause the processor to:

acquire detection results of the obstacles at multiple moments by utilizing the camera with the long focus lens and the camera with the short focus lens;

perform a target tracking to the acquired detection results, to obtain at least two tracking sequences, wherein each tracking sequence comprises detection results acquired at the multiple moments for a same obstacle; and

match two random tracking sequences of the at least two tracking sequences, and combining the two random tracking sequences into a combined tracking sequence, if the matching is successful.

6. The device according to claim 5 , wherein the computer programs are executed by the processor to cause the processor to further:

select the two random tracking sequences from the at least two tracking sequences;

acquire, for each moment, detection results from the selected two tracking sequences respectively, and project detection results acquired by the camera with the long focus lens onto an imaging plane acquired by the camera with the short focus lens, if the detection results from the selected two tracking sequences respectively are acquired by cameras with different focus lenses, and calculate an Intersection over Union (IoU) of the projected detection results and detection results acquired by the camera with the short focus lens;

determine an average of the IoUs calculated for the multiple moments; and

determine the matching of the selected two tracking sequences is successful if the average is greater than a preset threshold.

7. The device according to claim 5 , wherein the computer programs are executed by the processor to cause the processor to further:

combine detection results of the two random tracking sequences to obtain a combined tracking sequence if the matching is successful.

8. The device according to claim 5 , wherein the computer programs are executed by the processor to cause the processor to further:

acquire a new detection result, and determine whether the new detection result is correlated with the combined tracking sequence, creating a new tracking sequence if the new detection result is not correlated with the combined tracking sequence, wherein the new tracking sequence comprises the new detection result, or incorporate the new detection result into the combined tracking sequence if the new detection result is correlated with the combined tracking sequence.

9. A non-transitory computer readable storage medium, in which a computer program is stored, wherein the program, when executed by a processor, causes the processor to execute a method for multi-focal sensing of an obstacle, comprising:

acquiring detection results of obstacles at multiple moments by utilizing a camera with long focus lens and a camera with short focus lens;

performing a target tracking to the acquired detection results, to obtain at least two tracking sequences, wherein each tracking sequence comprises detection results acquired at the multiple moments for a same obstacle; and

matching two random tracking sequences of the at least two tracking sequences, and combining the two random tracking sequences into a combined tracking sequence, if the matching is successful.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE APPLICANT NAME PREVIOUSLY RECORDED AT REEL: 057933 FRAME: 0812. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Dec 28, 2021
From: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
To: APOLLO INTELLIGENT DRIVING TECHNOLOGY (BEIJING) CO., LTD.
Reel/Frame 058594/0836 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 13, 2021
From: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
To: APOLLO INTELLIGENT DRIVING (BEIJING) TECHNOLOGY CO., LTD.
Reel/Frame 057933/0812 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 11, 2019
From: GAO, HAN; LI, GUO; XUN, SUN; XIA, TIAN
To: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
Reel/Frame 049728/0213 →
Priority Claims (1)
CN 201811068019.4 · Sep 13, 2018 · national
Continuity (1)
Related Publication 20200089976A1 · Mar 19, 2020