Method of evaluating people's performance under high cognitive load
Methods and systems are herein providing for predicting driving performance based on mental state. In one example, a vehicle system comprises a vehicle computing system comprising one or more processors and memory storing instructions that, when executed by the one or more processors, cause the vehicle computing system to adjust one or more vehicle behaviors according to predicted driving performance, wherein the predicted driving performance is determined based on one or more mental states of a driver.
1 . A vehicle system, comprising:
a vehicle computing system comprising:
a driving performance evaluator, wherein the driving performance evaluator is built based on data obtained during one or more controlled environment scenarios, and wherein at least one of the one or more controlled environment scenario comprises a plurality of periods including a calibration period, a rest period, and a driving period, the driving period comprising city driving portions and highway driving portions; and
one or more processors and memory storing instructions that, when executed by the one or more processors, cause the vehicle computing system to:
determine a predicted driving performance via the driving performance evaluator; and
adjust one or more vehicle behaviors according to the predicted driving performance, wherein the predicted driving performance is determined based on one or more mental states of a driver.
2 . The vehicle system of claim 1 , wherein the one or more mental states of the driver are determined based on one or more biosignals acquired by one or more sensors of the vehicle system.
3 . The vehicle system of claim 2 , wherein the one or biosignals comprise one or more of eye movements, eyelid movements, and heart rate.
4 . The vehicle system of claim 2 , wherein the one or more controlled environment scenarios are provided by a vehicle simulation system configured to collect one or more second biosignals and vehicle telemetry information, and wherein one or more reference mental states are determined from the one or more second biosignals and driving performance is correlated based on the one or more reference mental states and the vehicle telemetry information.
5 . The vehicle system of claim 1 , wherein adjusting the one or more vehicle behaviors comprises one or more of adjusting one or more advanced driver-assistance system (ADAS) parameters and generating one or more notifications for presentation to the driver.
6 . A computer-implemented method for predicting driving performance based on one or more mental states, the method comprising:
collecting one or more first biosignals during a variable cognitive load vehicle environment scenario, wherein the one or more first biosignals are collected using a vehicle simulation system configured to provide a controlled environment;
collecting vehicle telemetry information during the variable cognitive load vehicle environment scenario with the vehicle simulation system;
determining one or more mental states based on the one or more first biosignals;
correlating the one or more mental states to the vehicle telemetry information to determine one or more driving performance metrics;
building a driving performance evaluator based on the one or more driving performance metrics; and
feeding one or more second biosignals acquired with a vehicle to the driving performance evaluator to predict driving performance, wherein the variable cognitive load vehicle environment scenario comprises a plurality of periods, including a calibration period, a rest period, and a driving period comprising city driving portions and highway driving portions.
7 . The computer-implemented method of claim 6 , wherein the one or more first biosignals are collected by one or more sensors, including one or more infrared (IR) sensors configured to track eye movements and eyelid movements.
8 . The computer-implemented method of claim 6 , wherein the one or more first biosignals comprise eye movements and eyelid movements.
9 . The computer-implemented method of claim 6 , wherein the vehicle telemetry information comprises vehicle speed, vehicle acceleration, position within lane, steering wheel position, and pedal position.
10 . The computer-implemented method of claim 6 , wherein the variable cognitive load vehicle environment scenario comprises a session wherein a user operates a vehicle simulation system to drive along a preprogrammed route while executing a variable cognitive load testing.
11 . The computer-implemented method of claim 10 , wherein the variable cognitive load testing comprises an NBACK test performed at various times while driving.
12 . A method of a vehicle computing system of a vehicle, comprising:
collecting one or more first biosignals of a driver of the vehicle with one or more first biosensors of the vehicle;
determining one or more mental states of the driver based on the one or more first biosignals;
feeding the one or more mental states into a driving performance evaluator, wherein the driving performance evaluator is a neural network trained on data obtained with a vehicle simulation system, and wherein the vehicle simulation system comprises second biosensors configured to acquire second biosignals of one or more users of the vehicle simulation system during variable cognitive load testing;
generating a predicted driving performance with the driving performance evaluator based on the one or more mental states; and
determining one or more vehicle interventions based on the predicted driving performance.
13 . The method of claim 12 , wherein the neural network of the driving performance evaluator is trained on one or more reference mental states and one or more driving performance metrics determined based on the second biosignals.
14 . The method of claim 13 , wherein the one or more second biosensors comprise one or more infrared (IR) eyetracking sensors.
15 . The method of claim 13 , wherein the one or more driving performance metrics and one or more reference mental states are generated by the vehicle simulation system, which is configured to provide a controlled environment for data collection.
16 . The method of claim 15 , wherein the vehicle simulation system is configured to establish various levels of cognitive load, wherein the one or more driving performance metrics are correlated to the various levels of cognitive load based on the one or more reference mental states and telemetry information of the vehicle simulation system.
17 . The method of claim 12 , wherein the one or more vehicle interventions comprise one or more of vehicle parameter adjustments and notifications presented to the driver via a vehicle infotainment unit.
18 . The computer-implemented method of claim 6 , wherein the driving performance evaluator is a neural network trained on the driving performance metrics, the one or more mental states, and/or one or more first biosignals, and wherein the computer-implemented method further comprises:
receiving driver input to one of confirm and correct predicted driving performance; and
updating training of the neural network based on the driver input.
19 . The method of claim 13 , wherein the one or more driving performance metrics are determined by correlating the one or more reference mental states with vehicle telemetry information of the vehicle simulation system, and wherein correlating the one or more reference mental states with the vehicle telemetry information comprises correlating driving behaviors with various amounts of cognitive load to indicate how reaction time depends on mental load.
20 . The method of claim 12 , wherein the vehicle simulation system comprises a user input device configured to receive inputs from the one or more users of the vehicle simulation system during the variable cognitive load testing.