IP Library Granted Patent US 11,120,275
Granted Patent B2
US 11,120,275 · App. 16/516,017 · Granted Sep 14, 2021

Visual perception method, apparatus, device, and medium based on an autonomous vehicle

Inventors: Jiajia Chen (Beijing, CN); Ji Wan (Beijing, CN); Tian Xia (Beijing, CN)
Assignee: Baidu Online Network Technology (Beijing) Co., Ltd.
G06K9/00791B60Q9/005G06N3/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,120,275
App. No.
16/516,017
Granted
Sep 14, 2021
Kind
B2
Abstract

The present disclosure provides a visual perception method, an apparatus, a device and a medium based on an autonomous vehicle, the method includes inputting an obtained first visual perception image collected by the autonomous vehicle into a first neural network model, recognizing multi-channel feature information of at least one target recognition object to be recognized, to eliminate redundant feature information in the first visual perception image; further, inputting the multi-channel feature information of the at least one target recognition object to be recognized into at least one sub-neural network model in a second neural network model respectively, to obtain at least one target recognition object; where there is a one to one correspondence between the target recognition object and the sub-neural network model. The present disclosure improves the speed of recognizing the target recognition object, thereby improving sensitivity of the autonomous vehicle and ensuring driving safety of the autonomous vehicle.

Claims (36)

1. A visual perception method based on an autonomous vehicle, comprising:

obtaining a first visual perception image collected by the autonomous vehicle;

inputting the first visual perception image into a first neural network model, and recognizing multi-channel feature information of at least one target recognition object to be recognized, to eliminate redundant feature information in the first visual perception image; and

inputting the multi-channel feature information of each of the at least one target recognition object to be recognized into a corresponding sub-neural network model of multiple sub-neural network models in a second neural network model, to obtain the at least one target recognition object; wherein each of the multiple sub-neural network models corresponds to a class of target recognition object.

2. The method according to claim 1 , wherein the method further comprises:

obtaining the first neural network model.

3. The method according to claim 2 , wherein the obtaining the first neural network model comprises:

training a first preset neural network model according to a second visual perception image and multi-channel feature information of at least one training sample object, to generate the first neural network model; wherein the second visual perception image comprises the at least one training sample object.

4. The method according to claim 1 , further comprises:

obtaining the second neural network model.

5. The method according to claim 4 , wherein the obtaining the second neural network model comprises:

training multiple sub-preset neural network models in a second preset neural network model according to at least one training sample object and multi-channel feature information of the at least one training sample object, to generate the second neural network model; wherein each of the multiple sub-preset neural network models corresponds to a class of training sample object.

6. The method according to claim 1 , wherein the obtaining the first visual perception image collected by the autonomous vehicle comprises:

obtaining the first visual perception image collected by an image collection apparatus in the autonomous vehicle.

7. The method according to claim 1 , wherein the target recognition object comprises at least one of the followings: a target substance and target semantics.

8. A visual perception apparatus based on an autonomous vehicle, comprising:

a processor; and

a computer-readable medium for storing program codes, which, when executed by the processor, the processor is caused to:

obtain a first visual perception image collected by the autonomous vehicle;

input the first visual perception image into a first neural network model, and recognize multi-channel feature information of at least one target recognition object to be recognized, to eliminate redundant feature information in the first visual perception image; and

input the multi-channel feature information of each of the at least one target recognition object to be recognized into a corresponding sub-neural network model of multiple sub-neural network models in a second neural network model, to obtain at least one target recognition object; wherein each of the multiple sub-neural network models corresponds to a class of target recognition object.

9. The apparatus according to claim 8 , wherein the processor is further caused to:

obtain the first neural network model.

10. The apparatus according to claim 9 , wherein the processor is further caused to:

train a first preset neural network model according to a second visual perception image and multi-channel feature information of at least one training sample object, to generate the first neural network model; wherein the second visual perception image comprises the at least one training sample object.

11. The apparatus according to claim 8 , wherein the processor is further caused to:

obtain the second neural network model.

12. The apparatus according to claim 11 , wherein the processor is further caused to:

train multiple sub-preset neural network models in a second preset neural network model according to at least one training sample object and multi-channel feature information of the at least one training sample object, to generate the second neural network model; wherein each of the multiple sub-preset neural network models corresponds to a class of training sample object.

13. The apparatus according to claim 8 , wherein the processor is further caused to:

obtain the first visual perception image collected by an image collection apparatus in the autonomous vehicle.

14. The apparatus according to claim 8 , wherein the target recognition object comprises at least one of the followings: a target substance and target semantics.

15. A non-transitory storage medium, comprising a non-transitory readable storage medium and a computer instruction, the computer instruction is stored in the readable storage medium; the computer instruction is configured to implement a visual perception method comprising the steps of:

obtaining a first visual perception image collected by the autonomous vehicle;

inputting the first visual perception image into a first neural network model, and recognizing multi-channel feature information of at least one target recognition object to be recognized, to eliminate redundant feature information in the first visual perception image; and

inputting the multi-channel feature information of each of the at least one target recognition object to be recognized into a corresponding sub-neural network model of multiple sub-neural network models in a second neural network model, to obtain the at least one target recognition object; wherein each of the multiple sub-neural network models corresponds to a class of target recognition object.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE APPLICANT NAME PREVIOUSLY RECORDED AT REEL: 057933 FRAME: 0812. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Dec 28, 2021
From: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
To: APOLLO INTELLIGENT DRIVING TECHNOLOGY (BEIJING) CO., LTD.
Reel/Frame 058594/0836 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 13, 2021
From: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
To: APOLLO INTELLIGENT DRIVING (BEIJING) TECHNOLOGY CO., LTD.
Reel/Frame 057933/0812 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 18, 2019
From: CHEN, JIAJIA; WAN, JI; XIA, TIAN
To: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
Reel/Frame 049795/0330 →
Priority Claims (1)
CN 201811055289.1 · Sep 11, 2018 · national
Continuity (1)
Related Publication 20200005051A1 · Jan 2, 2020