In-vehicle drowsiness analysis using blink rate
Disclosed techniques include in-vehicle drowsiness analysis using blink-rate. Video of an individual is obtained within a vehicle using an image capture device. The video is analyzed using one or more processors to detect a blink event based on a classifier for a blink that was determined. Using temporal analysis, the blink event is determined by identifying that eyes of the individual are closed for a frame in the video. Using the blink event and one or more other blink events, blink-rate information is determined using the one or more processors. Based on the blink-rate information, a drowsiness metric is calculated using the one or more processors. The vehicle is manipulated based on the drowsiness metric. A blink duration of the individual for the blink event is evaluated. The blink-rate information is compensated. The compensating is based on demographic information for the individual.
1. A computer-implemented method for mental state analysis comprising:
obtaining video of an individual within a vehicle with an image capture device;
analyzing, using one or more processors, the video to detect a blink event based on a classifier for a blink that was determined, wherein the blink event is determined by identifying that eyes of the individual are closed for a frame in the video using temporal analysis;
determining, using the one or more processors, blink-rate information using the blink event and one or more other blink events;
evaluating blinking for a group of people in the vehicle:
determining a difference in blinking between the individual and a remainder of the group:
calculating, using the one or more processors, a drowsiness metric, based on the blink-rate information and the difference in blinking; and
manipulating the vehicle, based on the drowsiness metric.
2. The method of claim 1 further comprising evaluating a blink duration of the individual for the blink event.
3. The method of claim 2 wherein a longer blink duration increases the drowsiness metric.
4. The method of claim 1 wherein the image capture device is located centrally in the vehicle's front interior.
5. The method of claim 4 wherein the image capture device is located above a shoulder line of the individual within the vehicle.
6. The method of claim 5 wherein the image capture device is attached to a rearview mirror of the vehicle.
7. The method of claim 1 wherein the determining yields a blink-rate frequency.
8. The method of claim 7 wherein a lower blink-rate frequency over time increases the drowsiness metric.
9. The method of claim 1 further comprising compensating the blink-rate information.
10. The method of claim 9 wherein the compensating is based on demographic information for the individual.
11. The method of claim 1 wherein the classifier is determined for a partially occluded face of the individual.
12. The method of claim 11 wherein the partially occluded face is a result of the individual wearing a face mask.
13. The method of claim 1 further comprising mapping the blink-rate information over a temporal distribution.
14. The method of claim 13 wherein the mapping is used in the calculating.
15. The method of claim 1 further comprising inferring mental states of the individual, wherein the mental states are based on the drowsiness metric.
16. The method of claim 1 further comprising determining a difference in blinking by the individual and typical blinking for the individual.
17. The method of claim 16 further comprising inferring mental states of the individual, wherein the mental states are based on the difference in blinking by the individual and typical blinking for the individual.
18. The method of claim 1 wherein the manipulating the vehicle includes recommending action.
19. The method of claim 1 wherein the manipulating the vehicle includes recommending content to the individual.
20. The method of claim 1 further comprising aggregating the blink-rate information for the individual with blink-rate information for a plurality of other people.
21. The method of claim 1 further comprising correlating the blink-rate information with activities being performed by the individual.
22. The method of claim 1 wherein the blink-rate information is correlated to a stimulus that the individual is encountering.
23. A computer program product embodied in a non-transitory computer readable medium for mental state analysis, the computer program product comprising code which causes one or more processors to perform operations of:
obtaining video of an individual within a vehicle with an image capture device;
analyzing, using one or more processors, the video to detect a blink event based on a classifier for a blink that was determined, wherein the blink event is determined by identifying that eyes of the individual are closed for a frame in the video using temporal analysis;
determining, using the one or more processors, blink-rate information using the blink event and one or more other blink events;
evaluating blinking for a group of people in the vehicle;
determining a difference in blinking between the individual and a remainder of the group;
calculating, using the one or more processors, a drowsiness metric, based on the blink-rate information and the difference in blinking; and
manipulating the vehicle, based on the drowsiness metric.
24. A computer system for mental state analysis comprising:
a memory which stores instructions;
one or more processors coupled to the memory wherein the one or more processors, when executing the instructions which are stored, are configured to:
obtain video of an individual within a vehicle with an image capture device;
analyze, using one or more processors, the video to detect a blink event based on a classifier for a blink that was determined, wherein the blink event is determined by identifying that eyes of the individual are closed for a frame in the video using temporal analysis;
determine, using the one or more processors, blink-rate information using the blink event and one or more other blink events;
evaluate blinking for a group of people in the vehicle;
determine a difference in blinking between the individual and a remainder of the group;
calculate, using the one or more processors, a drowsiness metric, based on the blink-rate information and the difference in blinking; and
manipulate the vehicle, based on the drowsiness metric.