Emotional adaptive driving policies for automated driving vehicles
In one example a system for emotional adaptive driving policies for automated driving vehicles, comprising a first plurality of sensors to detect environmental information relating to at least one passenger in a vehicle and a controller communicatively coupled to the plurality of sensors and comprising processing circuitry, to receive the environmental information from the first plurality of sensors, determine, from the environmental information, an emotional state of the at least one passenger, and implement a driving policy based at least in part on the emotional state of the at least one passenger. Other examples may be described.
1. A system for emotional adaptive driving policies for automated driving vehicles, comprising:
a first plurality of sensors configured to detect environmental information relating to at least one occupant in a vehicle; and
a controller communicatively coupled to the first plurality of sensors and comprising processing circuitry configured to:
receive the environmental information from the first plurality of sensors;
determine, from the environmental information, an emotional state of the at least one occupant; and
implement a driving policy based at least in part on the emotional state of the at least one occupant,
wherein the driving policy defines a set of environment observations including the environmental information, and a set of actions to be applied by an automated driving vehicle to execute driving maneuvers in response to road events in accordance with one or more control parameters, and
wherein the one or more control parameters are adjusted based upon the emotional state of the at least one occupant.
2. The system of claim 1 , wherein the first plurality of sensors comprises at least one of a camera sensor, a microphone sensor, or a biometric sensor.
3. The system of claim 2 , wherein the controller comprises processing circuitry configured to implement:
a heart rate emotion classifier to determine the emotional state of the at least one occupant from an input representative of the heart rate of the at least one occupant;
an audio emotion classifier to determine the emotional state of the at least one occupant from an input representative of an audio output of the at least one occupant;
a semantics emotion classifier to determine the emotional state of the at least one occupant from an input representative of semantics of an audio output of the at least one occupant; and
a computer vision emotion classifier to determine the emotional state of the at least one occupant from an output of the camera sensor.
4. The system of claim 3 , further comprising:
a second plurality of sensors configured to detect context information relating to the environment surrounding the vehicle,
wherein the second plurality of sensors comprises at least one of a video sensor, a RADAR sensor, a LIDAR sensor, an inertial measurement unit (IMU) sensor, or a global navigational satellite system (GNSS) sensor.
5. The system of claim 4 , wherein the controller comprises processing circuitry configured to implement:
a sensor based object recognition module to identify one or more objects proximate to the vehicle based on inputs from the second plurality of sensors; and
a localization module to determine a location of the vehicle based on inputs from at least one of the IMU sensor or the GNSS sensor.
6. The system of claim 5 , further comprising:
processing circuitry configured to implement a driving policy engine to generate a trajectory for the vehicle.
7. The system of claim 6 , wherein the driving policy engine comprises:
a semantic understanding module;
a trajectory generation module; and
a trajectory monitoring module.
8. The system of claim 6 , wherein the driving policy engine comprises:
a data store comprising emotion rules:
a data store comprising traffic rules;
a path planning module; and
a trajectory generation and monitoring module.
9. The system of claim 6 , wherein the driving policy engine executes logic, at least partially including hardware logic, to:
determine a current trajectory for the vehicle;
generate an updated trajectory for the vehicle based at least in part on an emotional state of the at least one occupant;
execute a maneuver based on the updated trajectory; and
alter the updated trajectory in response to a change in the emotional state of the at least one occupant.
10. The system of claim 6 , wherein the driving policy engine comprises:
a neural network model programmed to maximize a reward function based at least in part on the emotional state of the at least one occupant.
11. A vehicle control system comprising:
a first plurality of sensors configured to detect environmental information relating to at least one occupant in a vehicle; and
a controller communicatively coupled to the first plurality of sensors and comprising processing circuitry configured to:
receive the environmental information from the first plurality of sensors;
determine, from the environmental information, an emotional state of the at least one occupant; and
implement a driving policy based at least in part on the emotional state of the at least one occupant; and
a vehicle actuation module comprising a plurality of actuators configured to execute the driving policy,
wherein the driving policy defines a set of environment observations including the environmental information, and a set of actions to be applied by an automated driving vehicle to execute driving maneuvers in response to road events in accordance with one or more control parameters, and
wherein the one or more control parameters are adjusted based upon the emotional state of the at least one occupant.
12. The system of claim 11 , wherein the first plurality of sensors comprises at least one of a camera sensor, a microphone sensor, or a biometric sensor.
13. The system of claim 12 , wherein the controller comprises processing circuitry configured to implement:
a heart rate emotion classifier to determine the emotional state of the at least one occupant from an input representative of the heart rate of the at least one occupant;
an audio emotion classifier to determine the emotional state of the at least one occupant from an input representative of an audio output of the at least one occupant;
a semantics emotion classifier to determine the emotional state of the at least one occupant from an input representative of semantics of an audio output of the at least one occupant; and
a computer vision emotion classifier to determine the emotional state of the at least one occupant from an output of the camera sensor.
14. The system of claim 13 , further comprising:
a second plurality of sensors to detect context information relating to the environment surrounding the vehicle,
wherein the second plurality of sensors comprises at least one of a video sensor, a RADAR sensor, a LIDAR sensor, an inertial measurement unit (IMU) sensor, or a global navigational satellite system (GNSS) sensor.
15. The system of claim 14 , wherein the controller comprises processing circuitry configured to implement:
a sensor based object recognition module to identify one or more objects proximate to the vehicle based on inputs from the second plurality of sensors; and
a localization module to determine a location of the vehicle based on inputs from at least one of the IMU sensor or the GNSS sensor.
16. The system of claim 15 , further comprising:
processing circuitry configured to implement a driving policy engine to generate a trajectory for the vehicle.
17. The system of claim 16 , wherein the driving policy engine comprises:
a semantic understanding module;
a trajectory generation module; and
a trajectory monitoring module.
18. The system of claim 16 , wherein the driving policy engine comprises:
a data store comprising emotion rules:
a data store comprising traffic rules;
a path planning module; and
a trajectory generation and monitoring module.
19. The system of claim 16 , wherein the driving policy engine executes logic, at least partially including hardware logic, to:
determine a current trajectory for the vehicle;
generate an updated trajectory for the vehicle based at least in part on an emotional state of the at least one occupant;
execute a maneuver based on the updated trajectory; and
alter the updated trajectory in response to a change in the emotional state of the at least one occupant.
20. The system of claim 16 , wherein the driving policy engine comprises:
a neural network model programmed to maximize a reward function based at least in part on the emotional state of the at least one occupant.
21. The system of claim 1 , wherein the controller is further configured to adjust the driving policy by adjusting the one or more control parameters based upon the emotional state of the at least one occupant such that the same driving maneuver from among the set of driving maneuvers is executed differently for different respective emotional states of the occupant.
22. The system of claim 1 , wherein the environmental information received from the first plurality of sensors is associated with a plurality of vehicle occupants including the at least one occupant, and
wherein the emotional state of the at least one occupant is determined based upon (i) the reaction of the at least one occupant, and (ii) a reaction between the at least one occupant and at least another occupant from among the plurality of vehicle occupants.
23. The system of claim 1 , wherein the controller is further configured to extract, from the environmental information, content of speech spoken by the at least one occupant, and to determine an emotional state of the at least one occupant based upon the content of the speech.