IP Library Granted Patent US 11,003,926
Granted Patent B2
US 11,003,926 · App. 16/713,297 · Granted May 11, 2021

Method and apparatus for recognizing boundary of traffic sign

Inventor: Xu Duan (Beijing, CN)
Assignee: Baidu Online Network Technology (Beijing) Co., Ltd.
G06K9/00818G06K9/6215G06K9/6257G06K9/6262G06T7/12G06T2207/20081G06T2207/30252
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,003,926
App. No.
16/713,297
Granted
May 11, 2021
Kind
B2
Abstract

A method and an apparatus for recognizing a boundary of a traffic sign are provided. The method includes: acquiring at least one initial boundary point of a traffic sign in a to-be-processed image, the initial boundary point being used to mark a boundary of the traffic sign; dividing the to-be-processed image into at least one image segmentation area based on the at least one initial boundary point, the image segmentation area including at least one initial boundary point; importing, for an image segmentation area in the at least one image segmentation area, the image segmentation area into a pre-trained traffic sign boundary point recognition model to obtain an update boundary point corresponding to the image segmentation area; and determining a boundary point of the traffic sign based on all the update boundary point corresponding to the image segmentation area.

Claims (45)

1. A method for recognizing a boundary of a traffic sign, the method comprising:

acquiring at least one initial boundary point of the traffic sign in a to-be-processed image, the initial boundary point being used to mark the boundary of the traffic sign;

dividing the to-be-processed image into at least one image segmentation area based on the at least one initial boundary point, the image segmentation area including the at least one initial boundary point;

importing, for an image segmentation area in the at least one image segmentation area, the image segmentation area into a pre-trained traffic sign boundary point recognition model to obtain an update boundary point corresponding to the image segmentation area, the traffic sign boundary point recognition model being configured to obtain the update boundary point of the traffic sign based on the initial boundary point; and

determining the boundary point of the traffic sign based on all the update boundary point corresponding to the image segmentation area.

2. The method according to claim 1 , wherein the traffic sign boundary point recognition model is obtained by following training steps:

acquiring a plurality of sample images and a sample update boundary point of a traffic sign corresponding to each sample image of the plurality of sample images, wherein the sample image includes a traffic sign image segmentation area and a sample initial boundary point corresponding to the traffic sign image segmentation area; and

using each sample image of the plurality of sample images as an input, and using the sample update boundary point corresponding to each sample image of the plurality of sample images as an output, to perform training to obtain the traffic sign boundary point recognition model.

3. The method according to claim 2 , wherein the using each sample image of the plurality of sample images as an input, and using the sample update boundary point corresponding to each sample image of the plurality of sample images as an output, to perform training to obtain the traffic sign boundary point recognition model, comprises:

performing following training steps: inputting sequentially the traffic sign image segmentation area and the sample initial boundary point of each sample image of the plurality of sample images into an initial traffic sign boundary point recognition model, to obtain a prediction update boundary point corresponding to each sample image of the plurality of sample images, comparing the prediction update boundary point corresponding to each sample image of the plurality of sample images with the sample update boundary point corresponding to the sample image, to obtain a prediction accuracy rate of the initial traffic sign boundary point recognition model, determining whether the prediction accuracy rate is greater than a preset accuracy rate threshold, and if the prediction accuracy rate is greater than the preset accuracy rate threshold, determining the initial traffic sign boundary point recognition model as a trained traffic sign boundary point recognition model.

4. The method according to claim 3 , wherein the using each sample image of the plurality of sample images as an input, and using the sample update boundary point corresponding to each sample image of the plurality of sample images as an output, to perform training to obtain the traffic sign boundary point recognition model, comprises:

adjusting a parameter of the initial traffic sign boundary point recognition model, in response to the prediction accuracy rate being not greater than the preset accuracy rate threshold, and continuing to perform the training steps.

5. The method according to claim 4 , wherein the adjusting a parameter of the initial traffic sign boundary point recognition model, comprises:

calculating a loss function value of the initial traffic sign boundary point recognition model based on the prediction update boundary point and the sample update boundary point; and

adjusting the parameter of the initial traffic sign boundary point recognition model by the loss function value.

6. The method according to claim 2 , wherein the sample update boundary point is acquired by following steps:

selecting a set number of initial random points from the sample image;

selecting at least one update random point belonging to the traffic sign image segmentation area from the initial random points; and

calculating a distance between an update random point in the at least one update random point and the sample initial boundary point, and setting an update random point having the smallest distance as the sample update boundary point.

7. An apparatus for recognizing a boundary of a traffic sign, the apparatus comprising:

at least one processor; and

a memory storing instructions, wherein the instructions when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising:

acquiring at least one initial boundary point of the traffic sign in a to-be-processed image, the initial boundary point being used to mark the boundary of the traffic sign;

dividing the to-be-processed image into at least one image segmentation area based on the at least one initial boundary point, the image segmentation area including at least one initial boundary point;

importing, for an image segmentation area in the at least one image segmentation area, the image segmentation area into a pre-trained traffic sign boundary point recognition model to obtain an update boundary point corresponding to the image segmentation area, the traffic sign boundary point recognition model being configured to obtain the update boundary point of the traffic sign based on the initial boundary point; and

determining the boundary point of the traffic sign based on all the update boundary point corresponding to the image segmentation area.

8. The apparatus according to claim 7 , wherein the traffic sign boundary point recognition model is obtained by following training steps:

acquiring a plurality of sample images and a sample update boundary point of a traffic sign corresponding to each sample image of the plurality of sample images, wherein the sample image includes a traffic sign image segmentation area and a sample initial boundary point corresponding to the traffic sign image segmentation area; and

training to obtain the traffic sign boundary point recognition model, by using each sample image of the plurality of sample images as an input and using the sample update boundary point corresponding to each sample image of the plurality of sample images as an output.

9. The apparatus according to claim 8 , wherein training to obtain the traffic sign boundary point recognition model, by using each sample image of the plurality of sample images as an input and using the sample update boundary point corresponding to each sample image of the plurality of sample images as an outputs:

inputting sequentially the traffic sign image segmentation area and the sample initial boundary point of each sample image of the plurality of sample images into an initial traffic sign boundary point recognition model, to obtain a prediction update boundary point corresponding to each sample image of the plurality of sample images, comparing the prediction update boundary point corresponding to each sample image of the plurality of sample images with the sample update boundary point corresponding to the sample image, to obtain a prediction accuracy rate of the initial traffic sign boundary point recognition model, determining whether the prediction accuracy rate is greater than a preset accuracy rate threshold, and if the prediction accuracy rate is greater than the preset accuracy rate threshold, determining the initial traffic sign boundary point recognition model as a trained traffic sign boundary point recognition model.

10. The apparatus according to claim 9 , wherein the training to obtain the traffic sign boundary point recognition model, by using each sample image of the plurality of sample images as an input and using the sample update boundary point corresponding to each sample image of the plurality of sample images as an output:

adjusting a parameter of the initial traffic sign boundary point recognition model, in response to the prediction accuracy rate being not greater than the preset accuracy rate threshold, and return to the traffic sign boundary point recognition model training module.

11. The apparatus according to claim 10 , wherein the adjusting a parameter of the initial traffic sign boundary point recognition model, comprises:

calculating a loss function value of the initial traffic sign boundary point recognition model based on the prediction update boundary point and the sample update boundary point; and

adjusting the parameter of the initial traffic sign boundary point recognition model by the loss function value.

12. The apparatus according to claim 8 , wherein the sample update boundary point is acquired by following steps:

selecting a set number of initial random points from the sample image;

selecting at least one update random point belonging to the traffic sign image segmentation area from the initial random points; and

calculating a distance between an update random point in the at least one update random point and the sample initial boundary point, and setting an update random point having a smallest distance as the sample update boundary point.

13. A non-transitory computer readable medium, storing a computer program thereon, the program, when executed by a processor, causes the processor to perform operations, the operations comprising:

acquiring at least one initial boundary point of the traffic sign in a to-be-processed image, the initial boundary point being used to mark the boundary of the traffic sign;

dividing the to-be-processed image into at least one image segmentation area based on the at least one initial boundary point, the image segmentation area including the at least one initial boundary point;

importing, for an image segmentation area in the at least one image segmentation area, the image segmentation area into a pre-trained traffic sign boundary point recognition model to obtain an update boundary point corresponding to the image segmentation area, the traffic sign boundary point recognition model being configured to obtain the update boundary point of the traffic sign based on the initial boundary point; and

determining the boundary point of the traffic sign based on all the update boundary point corresponding to the image segmentation area.

Assignments (3)
CORRECTIVE ASSIGNMENT TO CORRECT THE APPLICANT NAME PREVIOUSLY RECORDED AT REEL: 057933 FRAME: 0812. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Dec 28, 2021
From: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
To: APOLLO INTELLIGENT DRIVING TECHNOLOGY (BEIJING) CO., LTD.
Reel/Frame 058594/0836 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 13, 2021
From: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
To: APOLLO INTELLIGENT DRIVING (BEIJING) TECHNOLOGY CO., LTD.
Reel/Frame 057933/0812 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Dec 13, 2019
From: DUAN, XU
To: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.
Reel/Frame 051288/0777 →
Priority Claims (1)
CN 201910435230.3 · May 23, 2019 · national
Continuity (1)
Related Publication 20200372268A1 · Nov 26, 2020