VIDEO-BASED DROWSY DRIVING DETECTION ON AN EDGE DEVICE

1 . A computer-implemented method for drowsy driving detection, the method comprising:

receiving, by at least one processor of a computing device in a vehicle, a first image at a first resolution from a camera in the vehicle;

processing, by the at least one processor, the first image to output a first facial image based on a predetermined position of a head of a driver of the vehicle in a field of view of the camera;

storing, by the at least one processor, the first facial image in a first buffer of a memory of the device;

selecting, by the at least one processor, a group of facial images from the first buffer, the group of facial images comprising the first facial image, wherein the group of facial images constitute a video of sequential facial images;

determining, by the at least one processor, a drowsy confidence score based on the video; and

generating, by the at least one processor, an alert if the drowsy confidence score is greater than a first threshold.

2 . The method of claim 1 , wherein determining the drowsy confidence score comprises:

processing the video, by a video-based neural network model in the at least one processor, to determine the drowsy confidence score.

3 . The method of claim 1 , further comprising:

storing, by the at least one processor, a second facial image in the first buffer, wherein the second facial image overwrites an earlier image that is not selected to form the video.

4 . The method of claim 2 , wherein the video-based neural network model is trained based on labels associated with videos of sequential facial images.

5 . The method of claim 1 , wherein the first image is one of continuous images captured by the camera, wherein the continuous images are captured at a first frame rate by the camera, wherein the first frame rate is greater than a predefined frame rate threshold, and wherein the first image is processed to output the first facial image in response to a determination that the first image corresponds to the predefined frame rate threshold.

6 . The method of claim 5 , further comprising:

sampling the continuous images to a second frame rate, the second frame rate being lower than the first frame rate.

7 . The method of claim 1 , further comprising:

resizing, by the at least one processor, the first image to a second resolution, wherein the second resolution is lower than the first resolution.

8 . The method of claim 7 , further comprising:

determining, by the at least one processor, an eye closure score of the driver of the vehicle based on the resized first image, wherein the generated alert indicates that the driver is drowsy, and wherein the generated alert is a severe alert if the eye closure score is greater than an eye closure threshold.

9 . The method of claim 7 , further comprising:

determining, by the at least one processor, a phone usage score of a driver based on the resized first image, wherein the generated alert indicates a phone usage alert if the phone usage score is greater than a phone usage threshold.

10 . The method of claim 7 , further comprising:

determining, by the at least one processor, a head pose score of a driver of the vehicle based on the resized first image,

wherein the generated alert is not sent to the driver if the head pose score is greater than a head pose threshold.

11 . The method of claim 7 , further comprising:

determining, by the at least one processor, a yawn score for a driver of the vehicle based on the resized first set of images,

wherein the generated alert is a moderate alert if the yawn score is greater than a yawn threshold.

12 . The method of claim 7 , wherein the first frame rate is at least 15 frames per second and the second frame rate is at least 5 frames per second, and wherein the second resolution is at least three times smaller than the first resolution.

13 . (canceled)

14 . The method of claim 1 , wherein the predetermined position of the head is based on a stored value.

15 . The method of claim 14 , wherein the stored value is a default position value.

16 . The method of claim 14 , further comprising:

detecting the position of the head of the driver in the field of view of the camera based on resized first image at the a second resolution; and

updating the stored value of the position of the head based on the detected position.

17 . The method of claim 16 , wherein generating an alert is further based on whether the stored value has not been updated within a threshold period of time.

18 . The method of claim 14 , wherein the stored value is from the last time that the device was turned on.

19 . The method of claim 14 , wherein the position of the head is configured at install time.

20 . The method of claim 14 , wherein the position of the head is determined by a pose detection model over multiple resized images of third resolution.

21 . The method of claim 1 , wherein processing the first image to output the first facial image comprises:

converting format of the first image from a first format to a second format, wherein the first format is YUV and the second format is BGR;

placing a bounding box of predefined dimensions on the first image at the predetermined position of the head;

cropping an image area enclosed by the bounding box, the image area comprising at least the face of the driver; and

resizing the cropped image area to output the first facial image with a third resolution.

22 . The method of claim 21 , wherein video-based neural network model is trained on labels assigned to cropped video clips, the cropped video clips comprising cropped video frames including at least heads of drivers.

23 . The method of claim 1 , further comprising:

sending, by the at least one processor via a communication interface, the severe alert to a driver device if the time elapsed from a previous alert sent to the driver device is greater than a first time threshold, wherein the first time threshold is defined by a fleet manager, wherein sending the severe alert comprises:

sending, by the at least one processor via the communication interface, the severe alert to a haptic device installed in the seat of the driver for generating vibrations based on alerts.

24 . (canceled)

25 . (canceled)

26 . (canceled)