Devices and methods for detecting drowsiness of drivers of vehicles
An apparatus includes: a camera configured to view a driver of a vehicle; and a processing unit comprising a first model and a second model; wherein the first model is configured to process the images of the driver and to output feature information based on the images, and wherein the second model is configured to receive the feature information from the first model, and to generate an output that indicates whether the driver is drowsy or not.
1 . An apparatus comprising:
a camera configured to view a driver of a vehicle, and to generate images of the driver; and
a processing unit comprising a first model and a second model;
wherein the first model is configured to process the images of the driver and to output feature information based on the images, and wherein the second model is configured to receive the feature information from the first model;
wherein the second model is configured to determine drowsiness information based on the feature information from the first model, and wherein the processing unit is configured to obtain information regarding an operation of the vehicle by the driver; and
wherein the information regarding the operation of the vehicle by the driver is independent of the images of the driver, and wherein the second model is downstream with respect to the first model that is configured to process the images of the driver.
2 . The apparatus of claim 1 , wherein the first model comprises a first neural network model configured to process the images.
3 . The apparatus of claim 2 , wherein the first neural network model is configured to identify one or more poses of the driver.
4 . The apparatus of claim 2 , wherein the second model comprises a second neural network model that is coupled in series with respect to the first neural network model.
5 . The apparatus of claim 4 , wherein the first neural network model is configured to identify poses of the driver over a duration, and output the feature information in a time series that indicates the identified poses of the driver over the duration; and
wherein the second neural network model is configured to process the time series to determine the drowsiness information.
6 . The apparatus of claim 4 , wherein the second neural network model is configured to receive the feature information in a time series.
7 . The apparatus of claim 1 , wherein the feature information includes information regarding eye closure of the driver.
8 . The apparatus of claim 1 , wherein the second model is configured to determine the drowsiness information based on a blink rate, a percentage-of-eyes-closed metric, a duration of eyes closed, time took to achieve eyelid closure, poses of the driver, or any combination of the foregoing.
9 . The apparatus of claim 1 , wherein the camera has a frame rate that is at least 10 frames per second.
10 . The apparatus of claim 1 , wherein the feature information cover features of the driver that occur over a duration that is at least: a fraction of a second, 1 second, 2 seconds, 5 seconds, 10 seconds, 12 seconds, 15 seconds, 20 seconds, 30 seconds, 1 minute, 2 minutes, 5 minutes, 10 minutes, 15 minutes, 20 minutes, 25 minutes, 30 minutes, or 40 minutes.
11 . The apparatus of claim 1 , wherein the processing unit is configured to obtain the information regarding the operation of the vehicle from a sensor.
12 . The apparatus of claim 11 , wherein the sensor is a part of the vehicle.
13 . The apparatus of claim 11 , wherein the sensor is a part of an on-board vehicle system.
14 . The apparatus of claim 1 , wherein the processing unit is configured to detect a micro sleep event.
15 . The apparatus of claim 1 , wherein the processing unit is configured to detect eyelid closure that occurs over a duration of sub-second, between 1 to 1.5 second, or more than 2 seconds.
16 . The apparatus of claim 1 , wherein the processing unit is configured to detect an early sign of drowsiness based the on images.
17 . The apparatus of claim 1 , wherein the drowsiness information indicates drowsiness or alertness.
18 . The apparatus of claim 1 , further comprising another camera configured to view an environment outside the vehicle, wherein the information regarding the operation of the vehicle comprises camera images provisioned by the other camera.
19 . The apparatus of claim 1 , wherein the information regarding the operation of the vehicle comprises acceleration information, deceleration information, information regarding turning of steering wheel or lack thereof, information regarding activating of turning light lever or lack thereof, information regarding changing of gear or lack thereof, information regarding braking or lack thereof, or information regarding pressing of acceleration pedal or lack thereof.