Method for evaluating state of driver, electronic device, and non-transitory computer readable storage medium

1 . A method for evaluating a state of a driver implemented by an electronic device, the method comprising:

performing, according to current driving scene data, driving behavior data and mental state data of the driver, a behavior analysis on the driver to obtain a behavior event of the driver, and generating temporal event information according to the behavior event;

performing, according to normal driving behavior data of the driver in a current time period, a driving habit analysis on the driver to generate driving habit information of the driver in the current time period;

invoking, according to identity information of the driver, a driver portrait library to obtain driver portrait information of the driver; and

performing, using a preset information fusion analysis model, an information fusion analysis on the temporal event information, the driving habit information and the driver portrait information, and evaluating the state of the driver according to an analysis result to obtain a state evaluation result of the driver.

2 . The method according to claim 1 , wherein said performing, using the preset information fusion analysis model, the information fusion analysis on the temporal event information, the driving habit information and the driver portrait information, and evaluating the state of the driver according to the analysis result to obtain the state evaluation result of the driver comprises:

performing, using a shallow fusion network in the information fusion analysis model, an early fusion analysis on the temporal event information, the driving habit information and the driver portrait information to generate an early fusion analysis vector;

performing, using a deep fusion network in the information fusion analysis model, a late fusion analysis on the early fusion analysis vector and the driver portrait information to generate a late fusion analysis vector; and

evaluating the state of the driver according to the late fusion analysis vector to obtain the state evaluation result of the driver, wherein the state evaluation result of the driver comprises one or more information selected from a group consisting of fatigue level information, distraction information, and driving risk level information.

3 . The method according to claim 2 , wherein said performing, using the shallow fusion network in the information fusion analysis model, the early fusion analysis on the temporal event information, the driving habit information and the driver portrait information to generate the early fusion analysis vector comprises:

inputting the temporal event information, which is used as a first input vector, into the shallow fusion network, and performing, using a behavior event analysis channel of the shallow fusion network, a multi-head self-attention analysis processing, a residual connection processing and a standardization processing on the temporal event information to generate a first analysis vector;

inputting the driving habit information, which is used as a second input vector, into the shallow fusion network, and performing, using a driving habit analysis channel of the shallow fusion network, the multi-head self-attention analysis processing, the residual connection processing and the standardization processing on the driving habit information to generate a second analysis vector;

inputting the driver portrait information, which is used as a third input vector, into the shallow fusion network, and performing, using a driver portrait analysis channel of the shallow fusion network, the multi-head self-attention analysis processing, the residual connection processing, the standardization processing, a collision detection processing and a logistic regression processing on the driver portrait information to generate a third analysis vector;

concatenating the first analysis vector and the second analysis vector to obtain a link vector; and

performing a dot product processing on the link vector and the third analysis vector to obtain the early fusion analysis vector.

4 . The method according to claim 2 , wherein said performing, using the deep fusion network in the information fusion analysis model, the late fusion analysis on the early fusion analysis vector and the driver portrait information to generate the late fusion analysis vector comprises:

inputting the early fusion analysis vector, which is used as a fourth input vector, into the deep fusion network, and performing, using a fusion result analysis channel in the deep fusion network, a multi-head self-attention analysis processing, a residual connection processing, a standardization processing and a concatenation processing on the early fusion analysis vector to generate a fourth analysis vector;

inputting the driver portrait information, which is used as a fifth input vector, into the deep fusion network, and performing, using a driver portrait analysis channel in the deep fusion network, the multi-head self-attention analysis processing, the residual connection processing, the standardization processing, a collision detection processing and a logistic regression processing on the driver portrait information to generate a fifth analysis vector; and

performing a dot product processing on the fourth analysis vector and the fifth analysis vector to obtain the late fusion analysis vector.

5 . The method according to claim 1 , wherein the temporal event information comprises at least one of information items selected from a group consisting of:

event category information, event-start-time-information, event-termination-time-information, event-start-speed-information, event-termination-speed-information, relative-collision-minimum-time-information, minimum-collision-distance-information, collision-accumulation-time-information, collision duration information, traffic flow information, vehicle position information, and vehicle driving duration information.

6 . The method according to claim 5 , wherein after said performing the behavior analysis on the driver according to the current driving scene data, the driving behavior data and the mental state data of the driver to obtain the behavior event of the driver, and generating the temporal event information according to the behavior event of the driver, the method further comprises:

obtaining real-time weather state information in a current driving scene, and adding the real-time weather state information which is taken as one of the information items, to the temporal event information.

7 . The method according to claim 5 , wherein after said performing the behavior analysis on the driver according to the current driving scene data, the driving behavior data and the mental state data of the driver to obtain the behavior event of the driver, and generating the temporal event information according to the behavior event of the driver, the method further comprises:

obtaining road information in a current driving scene, and adding the road information, which is taken as one of the information items, to the temporal event information.

8 . An electronic device, comprising a memory, a processor and a computer program stored in the memory and executable by the processor, wherein the processor is configured to, when executing the computer program, implement a method for evaluating a state of a driver, comprising steps of:

performing, according to current driving scene data, driving behavior data and mental state data of the driver, a behavior analysis on the driver to obtain a behavior event of the driver, and generating temporal event information according to the behavior event;

performing, according to normal driving behavior data of the driver in a current time period, a driving habit analysis on the driver to generate driving habit information of the driver in the current time period;

invoking, according to identity information of the driver, a driver portrait library to obtain driver portrait information of the driver; and

performing, using a preset information fusion analysis model, an information fusion analysis on the temporal event information, the driving habit information and the driver portrait information, and evaluating the state of the driver according to an analysis result to obtain a state evaluation result of the driver.

9 . The electronic device according to claim 8 , wherein the processor is further configured to perform the step of performing, using the preset information fusion analysis model, the information fusion analysis on the temporal event information, the driving habit information and the driver portrait information, and evaluating the state of the driver according to the analysis result to obtain the state evaluation result of the driver by:

performing, using a shallow fusion network in the information fusion analysis model, an early fusion analysis on the temporal event information, the driving habit information and the driver portrait information to generate an early fusion analysis vector;

performing, using a deep fusion network in the information fusion analysis model, a late fusion analysis on the early fusion analysis vector and the driver portrait information to generate a late fusion analysis vector; and

evaluating the state of the driver according to the late fusion analysis vector to obtain the state evaluation result of the driver, wherein the state evaluation result of the driver comprises one or more information selected from a group consisting of fatigue level information, distraction information, and driving risk level information.

10 . The electronic device according to claim 9 , wherein the processor is further configured to perform the step of performing, using the shallow fusion network in the information fusion analysis model, the early fusion analysis on the temporal event information, the driving habit information and the driver portrait information to generate the early fusion analysis vector by:

inputting the temporal event information, which is used as a first input vector, into the shallow fusion network, and performing, using a behavior event analysis channel of the shallow fusion network, a multi-head self-attention analysis processing, a residual connection processing and a standardization processing on the temporal event information to generate a first analysis vector;

inputting the driving habit information, which is used as a second input vector, into the shallow fusion network, and performing, using a driving habit analysis channel of the shallow fusion network, the multi-head self-attention analysis processing, the residual connection processing and the standardization processing on the driving habit information to generate a second analysis vector;

inputting the driver portrait information, which is used as a third input vector, into the shallow fusion network, and performing, using a driver portrait analysis channel of the shallow fusion network, the multi-head self-attention analysis processing, the residual connection processing, the standardization processing, a collision detection processing and a logistic regression processing on the driver portrait information to generate a third analysis vector;

concatenating the first analysis vector and the second analysis vector to obtain a link vector; and

performing a dot product processing on the link vector and the third analysis vector to obtain the early fusion analysis vector.

11 . The electronic device according to claim 9 , wherein the processor is further configured to perform the step of performing, using the deep fusion network in the information fusion analysis model, the late fusion analysis on the early fusion analysis vector and the driver portrait information to generate the late fusion analysis vector by:

inputting the early fusion analysis vector, which is used as a fourth input vector, into the deep fusion network, and performing, using a fusion result analysis channel in the deep fusion network, a multi-head self-attention analysis processing, a residual connection processing, a standardization processing and a concatenation processing on the early fusion analysis vector to generate a fourth analysis vector;

inputting the driver portrait information, which is used as a fifth input vector, into the deep fusion network, and performing, using a driver portrait analysis channel in the deep fusion network, the multi-head self-attention analysis processing, the residual connection processing, the standardization processing, a collision detection processing and a logistic regression processing on the driver portrait information to generate a fifth analysis vector; and

performing a dot product processing on the fourth analysis vector and the fifth analysis vector to obtain the late fusion analysis vector.

12 . The electronic device according to claim 8 , wherein the temporal event information comprises at least one of information items selected from a group consisting of event category information, event-start-time-information, event-termination-time-information, event-start-speed-information, event-termination-speed-information, relative-collision-minimum-time-information, minimum-collision-distance-information, collision-accumulation-time-information, collision duration information, traffic flow information, vehicle position information, and vehicle driving duration information.

13 . The electronic device according to claim 12 , wherein after the step of performing the behavior analysis on the driver according to the current driving scene data, the driving behavior data and the mental state data of the driver to obtain the behavior event of the driver, and generating the temporal event information according to the behavior event of the driver, the processor is further configured to perform a step of:

obtaining real-time weather state information in a current driving scene, and adding the real-time weather state information which is taken as one of the information items, to the temporal event information.

14 . The electronic device according to claim 12 , wherein after the step of performing the behavior analysis on the driver according to the current driving scene data, the driving behavior data and the mental state data of the driver to obtain the behavior event of the driver, and generating the temporal event information according to the behavior event of the driver, the processor is further configured to perform a step of:

obtaining road information in a current driving scene, and adding the road information, which is taken as one of the information items, to the temporal event information.

15 . A non-transitory computer readable storage medium which stores a computer program, that, when executed by a processor of an electronic device, causes the processor of the electronic device to implement a method for evaluating a state of a driver, comprising steps of:

performing, according to current driving scene data, driving behavior data and mental state data of the driver, a behavior analysis on the driver to obtain a behavior event of the driver, and generating temporal event information according to the behavior event;

performing, according to normal driving behavior data of the driver in a current time period, a driving habit analysis on the driver to generate driving habit information of the driver in the current time period;

invoking, according to identity information of the driver, a driver portrait library to obtain driver portrait information of the driver; and

performing, using a preset information fusion analysis model, an information fusion analysis on the temporal event information, the driving habit information and the driver portrait information, and evaluating the state of the driver according to an analysis result to obtain a state evaluation result of the driver.

16 . The non-transitory computer readable storage medium according to claim 15 , wherein the computer program is further configured to cause the processor of the electronic device to implement the step of performing, using the preset information fusion analysis model, the information fusion analysis on the temporal event information, the driving habit information and the driver portrait information, and evaluating the state of the driver according to the analysis result to obtain the state evaluation result of the driver by:

performing, using a shallow fusion network in the information fusion analysis model, an early fusion analysis on the temporal event information, the driving habit information and the driver portrait information to generate an early fusion analysis vector;

performing, using a deep fusion network in the information fusion analysis model, a late fusion analysis on the early fusion analysis vector and the driver portrait information to generate a late fusion analysis vector; and

evaluating the state of the driver according to the late fusion analysis vector to obtain the state evaluation result of the driver, wherein the state evaluation result of the driver comprises one or more information selected from a group consisting of fatigue level information, distraction information, and driving risk level information.

17 . The non-transitory computer readable storage medium according to claim 16 , wherein the computer program is further configured to cause the processor of the electronic device to implement the step of performing, using the shallow fusion network in the information fusion analysis model, the early fusion analysis on the temporal event information, the driving habit information and the driver portrait information to generate the early fusion analysis vector by:

inputting the temporal event information, which is used as a first input vector, into the shallow fusion network, and performing, using a behavior event analysis channel of the shallow fusion network, a multi-head self-attention analysis processing, a residual connection processing and a standardization processing on the temporal event information to generate a first analysis vector;

inputting the driving habit information, which is used as a second input vector, into the shallow fusion network, and performing, using a driving habit analysis channel of the shallow fusion network, the multi-head self-attention analysis processing, the residual connection processing and the standardization processing on the driving habit information to generate a second analysis vector;

inputting the driver portrait information, which is used as a third input vector, into the shallow fusion network, and performing, using a driver portrait analysis channel of the shallow fusion network, the multi-head self-attention analysis processing, the residual connection processing, the standardization processing, a collision detection processing and a logistic regression processing on the driver portrait information to generate a third analysis vector;

concatenating the first analysis vector and the second analysis vector to obtain a link vector; and

performing a dot product processing on the link vector and the third analysis vector to obtain the early fusion analysis vector.

18 . The non-transitory computer readable storage medium according to claim 16 , wherein the computer program is further configured to cause the processor of the electronic device to implement the step of performing, using the deep fusion network in the information fusion analysis model, the late fusion analysis on the early fusion analysis vector and the driver portrait information to generate the late fusion analysis vector by:

inputting the early fusion analysis vector, which is used as a fourth input vector, into the deep fusion network, and performing, using a fusion result analysis channel in the deep fusion network, a multi-head self-attention analysis processing, a residual connection processing, a standardization processing and a concatenation processing on the early fusion analysis vector to generate a fourth analysis vector;

inputting the driver portrait information, which is used as a fifth input vector, into the deep fusion network, and performing, using a driver portrait analysis channel in the deep fusion network, the multi-head self-attention analysis processing, the residual connection processing, the standardization processing, a collision detection processing and a logistic regression processing on the driver portrait information to generate a fifth analysis vector; and

performing a dot product processing on the fourth analysis vector and the fifth analysis vector to obtain the late fusion analysis vector.

19 . The non-transitory computer readable storage medium according to claim 15 , wherein after the step of performing the behavior analysis on the driver according to the current driving scene data, the driving behavior data and the mental state data of the driver to obtain the behavior event of the driver, and generating the temporal event information according to the behavior event of the driver, the computer program is further configured to cause the processor of the electronic device to perform a step of:

obtaining real-time weather state information in a current driving scene, and adding the real-time weather state information which is taken as one of the information items, to the temporal event information.

20 . The non-transitory computer readable storage medium according to claim 15 , wherein after the step of performing the behavior analysis on the driver according to the current driving scene data, the driving behavior data and the mental state data of the driver to obtain the behavior event of the driver, and generating the temporal event information according to the behavior event of the driver, the computer program is further configured to cause the processor of the electronic device to perform a step of:

obtaining road information in a current driving scene, and adding the road information, which is taken as one of the information items, to the temporal event information.