IP Library Granted Patent US 11,017,270
Granted Patent B2
US 11,017,270 · App. 16/566,824 · Granted May 25, 2021

Method and apparatus for image processing for vehicle

Inventors: Kun Tang (Beijing, CN); Hao Yu (Beijing, CN); Yongshan Yan (Beijing, CN); Chao Zheng (Beijing, CN); Yunfei Zhang (Beijing, CN); Yu Jiang (Beijing, CN)
Assignee: Baidu Online Network Technology (Beijing) Co., Ltd.
G06K9/6262G06K9/46G06K9/6255G06N3/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,017,270
App. No.
16/566,824
Granted
May 25, 2021
Kind
B2
Abstract

A method and an apparatus for image processing for a vehicle are provided. The method includes: acquiring an image presenting a target object; occluding at least one area of the image to obtain at least one occluded image; inputting the at least one occluded image into a pre-trained image processing model to obtain a driving parameter corresponding to each of the at least one occluded image; determining, for each driving parameter, a difference between the driving parameter and a real driving parameter corresponding to the acquired image; and determining whether the difference between the driving parameter of the occluded image occluding at least one area where the target object is located and the real driving parameter is greater than or equal to a difference threshold, to determine a degree of association between the state of the target object and the driving parameter.

Claims (40)

1. A method for image processing for a vehicle, the method comprising:

acquiring an image presenting a target object;

occluding at least one area of the acquired image to obtain at least one occluded image;

inputting the at least one occluded image into a pre-trained image processing model to obtain a driving parameter corresponding to each of the at least one occluded image, the pre-trained image processing model being configured to characterize a corresponding relationship between a state of the target object presented in the image and the obtained driving parameter of the vehicle;

determining, for each obtained driving parameter, a difference between the obtained driving parameter and a real driving parameter corresponding to the acquired image, wherein the real driving parameter is a driving parameter that is used by a driver when encountering the target object of the acquired image when driving the vehicle in a real scenario that the driver is driving; and

determining whether the difference between the driving parameter of the occluded image occluding the at least one area where the target object is located and the real driving parameter is greater than or equal to a difference threshold, to determine a degree of association between the state of the target object and the driving parameter, and to determine a learning status of the image processing model on a feature of the state of the target object.

2. The method for image processing for the vehicle according to claim 1 , wherein the driving parameter comprises a steering wheel angle and/or a driving speed.

3. The method for image processing for the vehicle according to claim 1 , wherein the method further comprises:

generating a thermodynamic diagram denoting the difference based on the difference between the driving parameter corresponding to the occluded image for each of the at least one occluded area of the acquired image and the real driving parameter.

4. The method for image processing for the vehicle according to claim 1 , wherein determining whether the difference between the driving parameter of the occluded image occluding the at least one area where the target object is located and the real driving parameter is greater than or equal to the difference threshold, to determine the degree of association between the state of the target object and the driving parameter comprises:

determining the degree of association between the state of the target object and the driving parameter being strong, in response to determining the difference between the driving parameter of the occluded image occluding the at least one area where the target object is located and the real driving parameter being greater than or equal to the difference threshold; and

determining the degree of association between the state of the target object and the driving parameter being weak, in response to determining the difference between the driving parameter of the occluded image occluding the at least one area where the target object is located and the real driving parameter being smaller than the difference threshold.

5. The method for image processing for the vehicle according to claim 1 , wherein before inputting the at least one occluded image into the pre-trained image processing model to obtain the driving parameter corresponding to each of the at least one occluded image, the method further comprises:

training a pre-established image processing model based on an end to end approach to obtain the pre-trained image processing model;

extracting a feature of the target object from at least one image presenting the target object; and

adding the extracted feature of the target object to a feature database of the pre-trained image processing model.

6. An image processing apparatus applied to a vehicle, the apparatus comprising:

at least one processor; and

a memory storing instructions, wherein the instructions when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising:

acquiring an image presenting a target object;

occluding at least one area of the acquired image to obtain at least one occluded image;

inputting the at least one occluded image into a pre-trained image processing model to obtain a driving parameter corresponding to each of the at least one occluded image, the pre-trained image processing model being configured to characterize a corresponding relationship between a state of the target object presented in the image and the obtained driving parameter of the vehicle;

determining, for each obtained driving parameter, a difference between the obtained driving parameter and a real driving parameter corresponding to the acquired image, wherein the real driving parameter is a driving parameter that is used by a driver when encountering the target object of the acquired image when driving the vehicle in a real scenario that the driver is driving; and

determining whether the difference between the driving parameter of the occluded image occluding the at least one area where the target object is located and the real driving parameter is greater than or equal to a difference threshold, to determine a degree of association between the state of the target object and the driving parameter and to determine a learning status of the image processing model on a feature of the state of the target object.

7. The image processing apparatus applied to the vehicle according to claim 6 , wherein the driving parameter comprises a steering wheel angle and/or a driving speed.

8. The image processing apparatus applied to a vehicle according to claim 6 , wherein the operations further comprise:

generating a thermodynamic diagram denoting the difference based on the difference between the driving parameter corresponding to the occluded image for each of the at least one occluded area of the acquired image and the real driving parameter.

9. The image processing apparatus applied to a vehicle according to claim 6 , wherein determining whether the difference between the driving parameter of the occluded image occluding the at least one area where the target object is located and the real driving parameter is greater than or equal to the difference threshold, to determine the degree of association between the state of the target object and the driving parameter comprises:

determining the degree of association between the state of the target object and the driving parameter being strong, in response to determining the difference between the driving parameter of the occluded image occluding the at least one area where the target object is located and the real driving parameter being greater than or equal to the difference threshold; and

determining the degree of association between the state of the target object and the driving parameter being weak, in response to determining the difference between the driving parameter of the occluded image occluding the at least one area where the target object is located and the real driving parameter being smaller than the difference threshold.

10. The image processing apparatus applied to a vehicle according to claim 6 , wherein the operations further comprise:

training a pre-established image processing model based on an end to end approach to obtain the pre-trained image processing model;

extracting a feature of the target object from at least one image presenting the target object; and

adding the extracted feature of the target object to a feature database of the pre-trained image processing model.

11. A non-transitory computer readable storage medium, storing a computer program thereon, wherein the computer program, when executed by a processor, causes the processor to perform operations, the operations comprising:

acquiring an image presenting a target object;

occluding at least one area of the image to obtain at least one occluded image;

inputting the at least one occluded image into a pre-trained image processing model to obtain a driving parameter corresponding to each of the at least one occluded image, the pre-trained image processing model being configured to characterize a corresponding relationship between a state of the target object presented in the image and the obtained driving parameter of a vehicle;

determining, for each obtained driving parameter, a difference between each obtained driving parameter and a real driving parameter corresponding to the acquired image, wherein the real driving parameter is a driving parameter that is used by a driver when encountering the target object of the acquired image when driving the vehicle in a real scenario that the driver is driving; and

determining whether the difference between the driving parameter of the occluded image occluding the at least one area where the target object is located and the real driving parameter is greater than or equal to a difference threshold, to determine a degree of association between the state of the target object and the driving parameter, and to determine a learning status of the image processing model on a feature of the state of the target object.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE APPLICANT NAME PREVIOUSLY RECORDED AT REEL: 057933 FRAME: 0812. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Dec 28, 2021
From: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
To: APOLLO INTELLIGENT DRIVING TECHNOLOGY (BEIJING) CO., LTD.
Reel/Frame 058594/0836 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 13, 2021
From: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
To: APOLLO INTELLIGENT DRIVING (BEIJING) TECHNOLOGY CO., LTD.
Reel/Frame 057933/0812 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 11, 2019
From: TANG, KUN; YU, HAO; YAN, YONGSHAN; ZHENG, CHAO; ZHANG, YUNFEI; JIANG, YU
To: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
Reel/Frame 050347/0729 →
Priority Claims (1)
CN 201710791373.9 · Sep 5, 2017 · national
Continuity (2)
Continuation PCTCN2018100248 · Aug 13, 2018
Related Publication 20200005091A1 · Jan 2, 2020