Context-sensitive adjustment of off-road glance time
Methods and systems involve obtaining information from one or more sources to determine a real-time context. A method includes determining a situational awareness score indicating a level of vigilance required based on the real-time context, obtaining images of eyes of a driver to detect a gaze pattern of the driver, determining a relevant attention score indicating a level of engagement of the driver in the real-time context based on a match between the gaze pattern of the driver and the real-time context, and obtaining images of the driver to detect behavior of the driver. A driver readiness score is determined and indicates a level of readiness of the driver to resume driving the vehicle based on the behavior of the driver. An off-road glance time is obtained based on using the situational awareness score, the relevant attention score, and the driver readiness score.
1. A method of adjusting an off-road glance time, which is a duration for which a driver using a hands-free mode of a semi-autonomous vehicle is allowed to be inattentive to a roadway, the method comprising:
obtaining information from one or more sources to determine a real-time context, the real-time context indicating at least a speed of the vehicle and a distance to a destination, wherein the one or more sources include one or more sensors of the vehicle;
determining a situational awareness score indicating a level of vigilance required for the driver to operate the semi-autonomous vehicle based on the real-time context;
obtaining images of eyes of the driver to detect a gaze pattern of the driver;
determining a relevant attention score indicating a level of engagement of the driver in the real-time context based on a match between the gaze pattern of the driver and the real-time context;
obtaining images of the driver to detect behavior of the driver;
determining a driver readiness score indicating a level of readiness of the driver to resume driving the semi-autonomous vehicle based on the behavior of the driver; and
adjusting the off-road glance time, which is the duration for which the driver using the hands-free mode of the semi-autonomous vehicle is allowed to be inattentive to the roadway, based on an adjustment within a range of duration values based on a weighted combination of the situational awareness score, the relevant attention score, and the driver readiness score.
2. The method according to claim 1 , wherein obtaining the information from the one or more sensors of the vehicle includes obtaining information from a camera, a radar system, a global navigation satellite system, or a lidar system.
3. The method according to claim 2 , wherein the obtaining the information from the one or more sources of the vehicle includes obtaining information from a communication device.
4. The method according to claim 3 , wherein the obtaining the information from the communication device includes obtaining traffic status or weather information.
5. The method according to claim 1 , wherein the determining the situational awareness score includes using a machine learning algorithm on the information from the one or more sources.
6. The method according to claim 1 , wherein the determining the relevant attention score includes using a neural network to determine the match between the gaze pattern of the driver and the real-time context.
7. The method according to claim 1 , wherein detecting the behavior of the driver includes classifying an activity of the driver and classifying a pose of the driver using convolutional neural networks.
8. The method according to claim 7 , wherein the determining the driver readiness score includes using a fully connected convolutional neural network on the activity of the driver and the pose of the driver.
9. The method according to claim 1 , further comprising starting a timer based on an off-road glance by the driver while the vehicle is in the hands-free mode.
10. The method according to claim 9 , further comprising providing an alert to the driver based on the timer reaching the off-road glance time.
11. A system to adjust an off-road glance time, which is a duration for which a driver using a hands-free mode of a semi-autonomous vehicle is allowed to be inattentive to a roadway, the system comprising:
one or more sources configured to provide information, wherein the one or more sources include one or more sensors of the vehicle;
a first camera within the vehicle configured to obtain images of eyes of the driver;
a second camera within the vehicle configured to obtain images of the driver; and
a controller configured to determine a real-time context based on the information, the real-time context indicating at least a speed of the vehicle and a distance to a destination, to determine a situational awareness score indicating a level of vigilance required for the driver to operate the semi-autonomous vehicle based on the real-time context, to detect a gaze pattern of the driver based on the images of the eyes of the driver, to determine a relevant attention score indicating a level of engagement of the driver in the real-time context based on a match between the gaze pattern of the driver and the real-time context, to detect behavior of the driver based on the images of the driver, to determine a driver readiness score indicating a level of readiness of the driver to resume driving the vehicle based on the behavior of the driver, and to adjust the off-road glance time, which is the duration for which the driver using the hands-free mode of the semi-autonomous vehicle is allowed to be inattentive to the roadway, based on an adjustment within a range of duration values based on a weighted combination of the situational awareness score, the relevant attention score, and the driver readiness score.
12. The system according to claim 11 , wherein the one or more sensors of the vehicle include a camera, a radar system, a global navigation satellite system, or a lidar system.
13. The system according to claim 12 , wherein the one or more sources of the vehicle include a communication device.
14. The system according to claim 13 , wherein the information from the communication device includes traffic status or weather information.
15. The system according to claim 11 , wherein the controller is configured to determine the situational awareness score by using a machine learning algorithm on the information from the one or more sources.
16. The system according to claim 11 , wherein the controller is configured to determine the relevant attention score by using a neural network to determine the match between the gaze pattern of the driver and the real-time context.
17. The system according to claim 11 , wherein the controller is configured to detect the behavior of the driver by classifying an activity of the driver and classifying a pose of the driver using convolutional neural networks.
18. The system according to claim 17 , wherein the controller is configured to determine the driver readiness score by using a fully connected convolutional neural network on the activity of the driver and the pose of the driver.
19. The system according to claim 11 , wherein the controller is additionally configured to start a timer based on an off-road glance by the driver while the vehicle is in the hands-free mode.
20. The system according to claim 19 , wherein the controller is additionally configured to provide an alert to the driver based on the timer reaching the off-road glance time.