System, method, and vehicle for recognition of traffic signs
A system and method for recognizing traffic signs during autonomous driving comprises a camera module obtaining a first recognition, a sensor obtains behavior information of the host vehicle and of nearby vehicles and a connected training module generates parameters for traffic sign recognition according to information as to the behavior of the host vehicle and the nearby vehicles. A recurrent neural network module is connected with the training module and the camera module, and outputs a second traffic sign recognition according to the parameters and the first recognition. Training parameters of the second recognition comprise the traffic sign recognition parameters and the first recognition, the accuracy of the training result being improved by combining other sensors to perform traffic sign recognition training.
1. A traffic sign recognition system applied to vehicle, the traffic sign recognition system comprising:
a camera module configured to obtain a first traffic sign recognition result;
a sensor configured to obtain a behavior information of a host vehicle and nearby vehicles;
a training module connecting to the sensor, the training module outputting traffic sign recognition parameters according to the behavior information of the host vehicle and the nearby vehicles; and
a recurrent neural network module connecting to the training module and the camera module,
wherein the recurrent neural network module outputs a second traffic sign recognition result according to the traffic sign recognition parameters and the first traffic sign recognition result,
the training module is a long-short term memory artificial neural network module, the sensor is further configured to obtain a driver behavior information of the host vehicle and the nearby vehicles, the long-short term memory artificial neural network module filters the driver behavior information to delete an information that adversely affects training,
the long-short term memory artificial neural network module comprises an information input layer and an information filtering layer, the information input layer is connected to the sensor to acquire the behavior information of the host vehicle and the nearby vehicles, and the information filtering layer outputs the traffic sign recognition parameters according to the behavior information of the host vehicle and the nearby vehicles, and
the information filtering layer comprises a gate structure, and the driver behavior information is added or deleted from a cellular state by the gate structure of the information filtering layer;
wherein the traffic sign recognition system further comprises a Kalman filter, the Kalman filter is connected to the recurrent neural network module, the Kalman filter is configured to fuse the first traffic sign recognition result and the second traffic sign recognition result to generate a third traffic sign recognition result;
if the behavior information of the host vehicle and nearby vehicles is abnormal, a weighting of vehicle behaviors in fusing the first traffic sign recognition result and the second traffic sign recognition result is reduced.
2. The traffic sign recognition system of claim 1 , wherein the traffic sign recognition parameters comprise:
location of vehicle center, yaw angle, speed, yaw rate, acceleration of the vehicle, width and height of the vehicle, and category of the vehicle.
3. The traffic sign recognition system of claim 1 , wherein the training module comprises a first convolutional neural network module and a second convolutional neural network module, the first convolutional neural network module is connected to the camera module and the recurrent neural network module, and the second convolutional neural network module is connected to the sensor and the recurrent neural network module,
the camera module obtains an image information,
the first convolutional neural network module generates a first traffic sign recognition result according to the image information; and
the second convolutional neural network module outputs the traffic sign recognition parameters according to the behavior information of the host vehicle and the nearby vehicles.
4. A traffic sign recognition method comprising:
obtaining a first traffic sign recognition result;
obtaining behavior information of a host vehicle and nearby vehicles;
outputting traffic sign recognition parameters according to the behavior information of the host vehicle and the nearby vehicles;
outputting a second traffic sign recognition result according to the traffic sign recognition parameters and the first traffic sign recognition result, and training parameters of the second traffic sign recognition result comprise the traffic sign recognition parameters and the first traffic sign recognition result; and
obtaining a driver behavior information of the host vehicle and the nearby vehicles, filtering the driver behavior information to delete an information that adversely affects training; and outputting the traffic sign recognition parameters according to the behavior information of the host vehicle and the nearby vehicles,
fusing the first traffic sign recognition result and the second traffic sign recognition result to generate a third traffic sign recognition result;
reducing a weighting of vehicle behaviors in fusing the first traffic sign recognition result and the second traffic sign recognition result if the behavior information of the host vehicle and nearby vehicles is abnormal;
wherein the driver behavior information is added or deleted from a cellular state by a gate structure of an information filtering layer.
5. The traffic sign recognition method of claim 4 , the traffic sign recognition parameters comprise:
location of vehicle center, yaw angle, speed, yaw rate, acceleration of the vehicle, width and height of the vehicle, and category of the vehicle.
6. The traffic sign recognition method of claim 5 , further comprising:
obtaining an image information;
outputting a first traffic sign recognition result according to the image information.
7. The traffic sign recognition method of claim 6 , further comprising:
outputting the traffic sign recognition parameters according to the behavior information of the host vehicle and the nearby vehicles.
8. The traffic sign recognition method of claim 4 , further comprising:
fusing the first traffic sign recognition result and the second traffic sign recognition result to generate a third traffic sign recognition result.
9. A vehicle comprising:
a traffic sign recognition system, the traffic sign recognition system comprising:
a camera module configured to obtain a first traffic sign recognition result;
a sensor configured to obtain behavior information of a host vehicle and nearby vehicles;
a training module connecting to the sensor, the training module outputting traffic sign recognition parameters according to the behavior information of the host vehicle and the nearby vehicles; and
a recurrent neural network module connecting to the training module and the camera module,
wherein the recurrent neural network module outputs a second traffic sign recognition result according to the traffic sign recognition parameters and the first traffic sign recognition result,
the training module is a long-short term memory artificial neural network module, the sensor is further configured to obtain a driver behavior information of the host vehicle and the nearby vehicles, the long-short term memory artificial neural network module filters the driver behavior information to delete an information that adversely affects training;
the long-short term memory artificial neural network module comprises an information input layer and an information filtering layer, the information input layer is connected to the sensor to acquire the behavior information of the host vehicle and the nearby vehicles, and the information filtering layer outputting outputs the traffic sign recognition parameters according to the behavior information of the host vehicle and the nearby vehicles, and
the information filtering layer comprises a gate structure, and the driver behavior information is added or deleted from a cellular state by the gate structure of the information filtering layer;
wherein the traffic sign recognition system further comprises a Kalman filter, the Kalman filter is connected to the recurrent neural network module, the Kalman filter is configured to fuse the first traffic sign recognition result and the second traffic sign recognition result to generate a third traffic sign recognition result;
if the behavior information of the host vehicle and nearby vehicles is abnormal, a weighting of vehicle behaviors in fusing the first traffic sign recognition result and the second traffic sign recognition result is reduced.
10. The vehicle of claim 9 , wherein the traffic sign recognition parameters comprise:
position of vehicle center, yaw angle, speed, yaw rate, acceleration of the vehicle, width and height of the vehicle, and category of the vehicle.
11. The vehicle of claim 9 , wherein the training module comprises a first convolutional neural network module and a second convolutional neural network module, the first convolutional neural network module is connected to the camera module and the recurrent neural network module, and the second convolutional neural network module is connected to the sensor and the recurrent neural network module,
the camera module obtains an image information,
the first convolutional neural network module generates a first traffic sign recognition result according to the image information, and
the second convolutional neural network module outputs the traffic sign recognition parameters according to the behavior information of the host vehicle and the nearby vehicles.