Intersection traffic control method, apparatus and system
An intersection traffic control method, apparatus and system are provided. The method includes that: a vehicle signal of a first vehicle at an intersection and a vehicle signal of a second vehicle located in a set zone in proximity to the intersection are acquired; the vehicle signal of the first vehicle and the vehicle signal of the second vehicle are input into an instruction learning model trained in advance based on a reinforcement learning principle, and a score of a preset traffic indicator of the first vehicle after executing a respective candidate action instruction is calculated; a reward of the first vehicle when executing the respective candidate action instruction is acquired according to the score of the preset traffic indicator, a candidate action instruction corresponding to a maximum reward is determined as an output result of the instruction learning model, and a next action instruction is determined according to the output result; and navigation of the first vehicle through the intersection is controlled according to the next action instruction.
1. An intersection traffic control method, comprising:
acquiring a vehicle signal of a first vehicle at an intersection and a vehicle signal of a second vehicle located in a set zone in proximity to the intersection;
inputting the vehicle signal of the first vehicle and the vehicle signal of the second vehicle into an instruction learning model trained in advance based on a reinforcement learning principle, acquiring an output result of the instruction learning model, and determining a next action instruction of the first vehicle according to the output result; and
controlling navigation of the first vehicle through the intersection according to the next action instruction,
wherein the instruction learning model calculates, according to the input vehicle signal of the first vehicle and the input vehicle signal of the second vehicle, a score of a preset traffic indicator of the first vehicle after executing a respective candidate action instruction, acquires, according to the score of the preset traffic indicator, a reward of the first vehicle when executing the respective candidate action instruction, and determines a candidate action instruction corresponding to a maximum reward as the output result;
the preset traffic indicator comprises a first traffic indicator acquired based on a speed, a second traffic indicator acquired based on a danger zone, and a third traffic indicator acquired based on an acceleration and a steering angle;
a score of the first traffic indicator is acquired by means of the following manner: determining the score of the first traffic indicator under the respective candidate action instruction according to an average speed of the first vehicle from entering the intersection to executing the respective candidate action instruction, the first traffic indicator being used to represent efficiency of the first vehicle passing through the intersection;
a score of the second traffic indicator is calculated by means of the following manner: determining the score of the second traffic indicator under the respective candidate action instruction according to an area of the danger zone between the first vehicle, when executing the respective candidate action instruction, and the second vehicle, the second traffic indicator being used to represent safety of the first vehicle passing through the intersection, and the danger zone being an overlapping zone of an elliptical zone where the first vehicle is located and an elliptical zone where the second vehicle is located;
a score of the third traffic indicator is calculated by means of the following manner: determining the score of the third traffic indicator under the respective candidate action instruction according to an acceleration and a steering wheel angle of the first vehicle and a time span taken by the first vehicle to pass through the intersection when the first vehicle executes the respective candidate action instruction, the third traffic indicator being used to represent stationarity of the first vehicle passing through the intersection;
the score of the second traffic indicator is acquired by the following formula:
f ( D )=0.25×[θ 1 ×( r 1 ) 2 +θ 2 ×( r 2 ) 2 −( h 1 +h 2 )× d 12 ],
where d 12 is a distance between a geometric center of the first vehicle and a geometric center of the second vehicle, r 1 and r 2 are a dynamic radius of the first vehicle in polar coordinates and a dynamic radius of the second vehicle in polar coordinates, respectively, θ 1 and θ 2 are an angle formed between the geometric center of the first vehicle and overlapping intersections of the overlapping zone in the polar coordinates and an angle formed between the geometric center of the second vehicle and the overlapping intersections of the overlapping zone in the polar coordinates, respectively, and h 1 and h 2 are vertical distances from the overlapping intersections of the overlapping zone to d 12 , respectively;
and/or,
the score of the third traffic indicator is acquired by the following formula:
f
(
α
,
θ
)
=
C
1
×
1
n
∑
i
=
1
n
(
d
α
dt
)
2
+
C
2
×
1
n
∑
i
=
1
n
(
d
θ
dt
)
2
,
where C 1 and C 2 are preset weight factors, n is the time span taken by the first vehicle to pass through the intersection, α is the acceleration of the first vehicle when executing the respective candidate action instruction, and θ is the steering wheel angle of the first vehicle when executing the respective candidate action instruction.
2. The intersection traffic control method as claimed in claim 1 , wherein
the reward is calculated by means of the following manner: performing weighted summation on the score of the first traffic indicator, the score of the second traffic indicator and the score of the third traffic indicator to acquire the reward of the first vehicle when executing the respective candidate action instruction,
wherein the vehicle signal comprises one or more of position information, speed information, acceleration information and steering wheel angle information of a vehicle.
3. The intersection traffic control method as claimed in claim 2 , wherein
the reward of the first vehicle when executing the respective candidate action instruction is acquired by the following formula:
Reward= k 1 ×f ( v )− k 2 ×f ( D )− k 3 ×f (α,θ),
where Reward represents the reward, f(v) is the first traffic indicator, f(D) is the second traffic indicator, f(α, θ) is the third traffic indicator, and k 1 , k 2 and k 3 are preset weight factors.
4. The intersection traffic control method as claimed in claim 1 , further comprising:
judging whether the first vehicle leaves the intersection, and if not, periodically acquiring the vehicle signal of the first vehicle and the vehicle signal of the second vehicle according to a preset frequency and controlling the navigation of the first vehicle based on the instruction learning model until the first vehicle passes through the intersection.
5. The intersection traffic control method as claimed in claim 4 , wherein
the intersection comprises any one of a roundabout intersection, a crossroad without a traffic light, and a T-shaped intersection without a traffic light;
and/or,
the first vehicle acquires the vehicle signal of the second vehicle located in the set zone in proximity to the intersection through Vehicle-to-Vehicle (V2V) communication.
6. The intersection traffic control method as claimed in claim 5 , wherein the instruction learning model is acquired by the following manner:
constructing a simulated intersection environment according to an actual intersection environment;
acquiring a plurality of sets of preset sample data, each set of preset sample data comprising a vehicle signal of a first vehicle and a vehicle signal of a second vehicle in the simulated intersection environment, and further comprising information of an optimal actual next action instruction enabling the first vehicle to normally pass through the intersection; and
inputting the sample data to the instruction learning model in the simulated intersection environment to calculate an indicator value and a reward to acquire a candidate action instruction, and modifying model parameter values of the instruction learning model according to a matching situation of the calculated candidate action instruction and the optimal actual next action instruction until a matching degree between the next action instruction of the first vehicle predicted by the instruction learning model and the optimal actual next action instruction of the first vehicle reaches a set matching threshold.
7. An intersection traffic control apparatus, comprising:
a signal acquisition module, configured to acquire a vehicle signal of a first vehicle at an intersection and a vehicle signal of a second vehicle located in a set zone in proximity to the intersection;
an instruction determination module, configured to input the vehicle signal of the first vehicle and the vehicle signal of the second vehicle into an instruction learning model trained in advance based on a reinforcement learning principle, acquire an output result of the instruction learning model, and determine a next action instruction of the first vehicle according to the output result; and
an execution module, configured to control navigation of the first vehicle through the intersection according to the next action instruction,
wherein the instruction learning model calculates, according to the input vehicle signal of the first vehicle and the input vehicle signal of the second vehicle, a score of a preset traffic indicator of the first vehicle after executing a respective candidate action instruction, acquires, according to the score of the preset traffic indicator, a reward of the first vehicle when executing the respective candidate action instruction, and determines a candidate action instruction corresponding to a maximum reward as the output result;
the preset traffic indicator comprises a first traffic indicator acquired based on a speed, a second traffic indicator acquired based on a danger zone, and a third traffic indicator acquired based on an acceleration and a steering angle;
a score of the first traffic indicator is acquired by means of the following manner: determining the score of the first traffic indicator under the respective candidate action instruction according to an average speed of the first vehicle from entering the intersection to executing the respective candidate action instruction, the first traffic indicator being used to represent efficiency of the first vehicle passing through the intersection;
a score of the second traffic indicator is calculated by means of the following manner: determining the score of the second traffic indicator under the respective candidate action instruction according to an area of the danger zone between the first vehicle, when executing the respective candidate action instruction, and the second vehicle, the second traffic indicator being used to represent safety of the first vehicle passing through the intersection, and the danger zone being an overlapping zone of an elliptical zone where the first vehicle is located and an elliptical zone where the second vehicle is located;
a score of the third traffic indicator is calculated by means of the following manner: determining the score of the third traffic indicator under the respective candidate action instruction according to an acceleration and a steering wheel angle of the first vehicle and a time span taken by the first vehicle to pass through the intersection when the first vehicle executes the respective candidate action instruction, the third traffic indicator being used to represent stationarity of the first vehicle passing through the intersection;
the score of the second traffic indicator is acquired by the following formula:
f ( D )=0.25×[θ 1 ×( r 1 ) 2 +θ 2 ×( r 2 ) 2 −( h 1 +h 2 )× d 12 ],
where d 12 is a distance between a geometric center of the first vehicle and a geometric center of the second vehicle, r 1 and r 2 are a dynamic radius of the first vehicle in polar coordinates and a dynamic radius of the second vehicle in polar coordinates, respectively, θ 1 and θ 2 are an angle formed between the geometric center of the first vehicle and overlapping intersections of the overlapping zone in the polar coordinates and an angle formed between the geometric center of the second vehicle and the overlapping intersections of the overlapping zone in the polar coordinates, respectively, and h 1 and h 2 are vertical distances from the overlapping intersections of the overlapping zone to d 12 , respectively;
and/or,
the score of the third traffic indicator is acquired by the following formula:
f
(
α
,
θ
)
=
C
1
×
1
n
∑
i
=
1
n
(
d
α
dt
)
2
+
C
2
×
1
n
∑
i
=
1
n
(
d
θ
dt
)
2
,
where C 1 and C 2 are preset weight factors, n is the time span taken by the first vehicle to pass through the intersection, α is the acceleration of the first vehicle when executing the respective candidate action instruction, and θ is the steering wheel angle of the first vehicle when executing the respective candidate action instruction.
8. An intersection traffic control system, comprising: a sensor module, a processor module and an executor module, wherein
the sensor module comprises a sensor, a Controller Area Network (CAN) receiver and a Vehicle-to-Vehicle Communication (V2V) signal transceiver, the sensor is configured to acquire vehicle information of a first vehicle at an intersection, the V2V signal transceiver is configured to acquire a vehicle signal of a second vehicle located in a set zone in proximity to the intersection, and the CAN receiver is configured to collect the vehicle signal of the first vehicle acquired by the sensor and the vehicle signal of the second vehicle acquired by the V2V signal transceiver, and send the vehicle signal of the first vehicle and the vehicle signal of the second vehicle to the processor module;
the processor module is configured to input the vehicle signal of the first vehicle and the vehicle signal of the second vehicle into an instruction learning model trained in advance based on a reinforcement learning principle, acquire an output result of the instruction learning model, determine a next action instruction according to the output result, and output the next action instruction to the executor module; and
the executor module is configured to control navigation of the first vehicle through the intersection according to the next action instruction,
wherein the instruction learning model calculates, according to the input vehicle signal of the first vehicle and the input vehicle signal of the second vehicle, a score of a preset traffic indicator of the first vehicle after executing a respective candidate action instruction, acquires, according to the score of the preset traffic indicator, a reward of the first vehicle when executing the respective candidate action instruction, and determines a candidate action instruction corresponding to a maximum reward as the output result;
the preset traffic indicator comprises a first traffic indicator acquired based on a speed, a second traffic indicator acquired based on a danger zone, and a third traffic indicator acquired based on an acceleration and a steering angle;
a score of the first traffic indicator is acquired by means of the following manner: determining the score of the first traffic indicator under the respective candidate action instruction according to an average speed of the first vehicle from entering the intersection to executing the respective candidate action instruction, the first traffic indicator being used to represent efficiency of the first vehicle passing through the intersection;
a score of the second traffic indicator is calculated by means of the following manner: determining the score of the second traffic indicator under the respective candidate action instruction according to an area of the danger zone between the first vehicle, when executing the respective candidate action instruction, and the second vehicle, the second traffic indicator being used to represent safety of the first vehicle passing through the intersection, and the danger zone being an overlapping zone of an elliptical zone where the first vehicle is located and an elliptical zone where the second vehicle is located;
a score of the third traffic indicator is calculated by means of the following manner: determining the score of the third traffic indicator under the respective candidate action instruction according to an acceleration and a steering wheel angle of the first vehicle and a time span taken by the first vehicle to pass through the intersection when the first vehicle executes the respective candidate action instruction, the third traffic indicator being used to represent stationarity of the first vehicle passing through the intersection;
the score of the second traffic indicator is acquired by the following formula:
f ( D )=0.25×[θ 1 ×( r 1 ) 2 +θ 2 ×( r 2 ) 2 −( h 1 +h 2 )× d 12 ],
where d 12 is a distance between a geometric center of the first vehicle and a geometric center of the second vehicle, r 1 and r 2 are a dynamic radius of the first vehicle in polar coordinates and a dynamic radius of the second vehicle in polar coordinates, respectively, θ 1 and θ 2 are an angle formed between the geometric center of the first vehicle and overlapping intersections of the overlapping zone in the polar coordinates and an angle formed between the geometric center of the second vehicle and the overlapping intersections of the overlapping zone in the polar coordinates, respectively, and h 1 and h 2 are vertical distances from the overlapping intersections of the overlapping zone to d 12 , respectively;
and/or,
the score of the third traffic indicator is acquired by the following formula:
f
(
α
,
θ
)
=
C
1
×
1
n
∑
i
=
1
n
(
d
α
dt
)
2
+
C
2
×
1
n
∑
i
=
1
n
(
d
θ
dt
)
2
,
where C 1 and C 2 are preset weight factors, n is the time span taken by the first vehicle to pass through the intersection, α is the acceleration of the first vehicle when executing the respective candidate action instruction, and θ is the steering wheel angle of the first vehicle when executing the respective candidate action instruction.
9. A computer-readable storage medium on which a computer program is stored, wherein the program is executed by a processor to implement operations in the intersection traffic control method as claimed in claim 1 .
10. A computer device, comprising a memory, a processor and a computer program that is stored on the memory and runnable on the processor, wherein the program is executed by the processor to implement operations in the intersection traffic control method as claimed in claim 1 .
11. The intersection traffic control method as claimed in claim 2 , further comprising:
judging whether the first vehicle leaves the intersection, and if not, periodically acquiring the vehicle signal of the first vehicle and the vehicle signal of the second vehicle according to a preset frequency and controlling the navigation of the first vehicle based on the instruction learning model until the first vehicle passes through the intersection.
12. The intersection traffic control method as claimed in claim 11 , wherein
the intersection comprises any one of a roundabout intersection, a crossroad without a traffic light, and a T-shaped intersection without a traffic light;
and/or,
the first vehicle acquires the vehicle signal of the second vehicle located in the set zone in proximity to the intersection through Vehicle-to-Vehicle (V2V) communication.
13. The intersection traffic control method as claimed in claim 12 , wherein the instruction learning model is acquired by the following manner:
constructing a simulated intersection environment according to an actual intersection environment;
acquiring a plurality of sets of preset sample data, each set of preset sample data comprising a vehicle signal of a first vehicle and a vehicle signal of a second vehicle in the simulated intersection environment, and further comprising information of an optimal actual next action instruction enabling the first vehicle to normally pass through the intersection; and
inputting the sample data to the instruction learning model in the simulated intersection environment to calculate an indicator value and a reward to acquire a candidate action instruction, and modifying model parameter values of the instruction learning model according to a matching situation of the calculated candidate action instruction and the optimal actual next action instruction until a matching degree between the next action instruction of the first vehicle predicted by the instruction learning model and the optimal actual next action instruction of the first vehicle reaches a set matching threshold.
14. The intersection traffic control method as claimed in claim 3 , further comprising:
judging whether the first vehicle leaves the intersection, and if not, periodically acquiring the vehicle signal of the first vehicle and the vehicle signal of the second vehicle according to a preset frequency and controlling the navigation of the first vehicle based on the instruction learning model until the first vehicle passes through the intersection.
15. The intersection traffic control method as claimed in claim 14 , wherein
the intersection comprises any one of a roundabout intersection, a crossroad without a traffic light, and a T-shaped intersection without a traffic light;
and/or,
the first vehicle acquires the vehicle signal of the second vehicle located in the set zone in proximity to the intersection through Vehicle-to-Vehicle (V2V) communication.
16. The intersection traffic control method as claimed in claim 15 , wherein the instruction learning model is acquired by the following manner:
constructing a simulated intersection environment according to an actual intersection environment;
acquiring a plurality of sets of preset sample data, each set of preset sample data comprising a vehicle signal of a first vehicle and a vehicle signal of a second vehicle in the simulated intersection environment, and further comprising information of an optimal actual next action instruction enabling the first vehicle to normally pass through the intersection; and
inputting the sample data to the instruction learning model in the simulated intersection environment to calculate an indicator value and a reward to acquire a candidate action instruction, and modifying model parameter values of the instruction learning model according to a matching situation of the calculated candidate action instruction and the optimal actual next action instruction until a matching degree between the next action instruction of the first vehicle predicted by the instruction learning model and the optimal actual next action instruction of the first vehicle reaches a set matching threshold.
17. The intersection traffic control apparatus as claimed in claim 7 , wherein the apparatus is further configured to:
judge whether the first vehicle leaves the intersection, and if not, periodically acquire the vehicle signal of the first vehicle and the vehicle signal of the second vehicle according to a preset frequency and controlling the navigation of the first vehicle based on the instruction learning model until the first vehicle passes through the intersection.
18. The intersection traffic control apparatus as claimed in claim 17 , wherein
the intersection comprises any one of a roundabout intersection, a crossroad without a traffic light, and a T-shaped intersection without a traffic light;
and/or,
the first vehicle acquires the vehicle signal of the second vehicle located in the set zone in proximity to the intersection through Vehicle-to-Vehicle (V2V) communication.
19. The intersection traffic control system as claimed in claim 8 , wherein the system is further configured to:
judge whether the first vehicle leaves the intersection, and if not, periodically acquire the vehicle signal of the first vehicle and the vehicle signal of the second vehicle according to a preset frequency and controlling the navigation of the first vehicle based on the instruction learning model until the first vehicle passes through the intersection.
20. The intersection traffic control system as claimed in claim 19 , wherein
the intersection comprises any one of a roundabout intersection, a crossroad without a traffic light, and a T-shaped intersection without a traffic light;
and/or,
the first vehicle acquires the vehicle signal of the second vehicle located in the set zone in proximity to the intersection through Vehicle-to-Vehicle (V2V) communication.