Circadian rhythm-based training data correction for drowsiness detection systems and applications
In various examples, circadian rhythm-based data augmentation for drowsiness detection systems and applications are provided. Embodiments described herein may produce an estimated circadian rhythm for a test subject and/or vehicle driver or other machine operator or occupant, and use the pattern of that circadian rhythm to correct, confirm, calibrate, or otherwise augment drowsiness assessments derived from video image data. The position of a person in the context of their process C circadian cycle may be used as indication of their level of drowsiness. An estimated process C circadian cycle may be used to generate more accurate ground truth training data for training machine learning models, and may be used by real-time, in-vehicle drowsiness detection systems that infer driver drowsiness levels based on captured images. In various embodiments, a circadian rhythm drowsiness estimate may be used to correct, calibrate, augment, and/or replace a drowsiness score predicted by a machine learning model.
1 . One or more processors comprising one or more processing units to:
receive a first ground truth image sequence representing visual characteristics of a test subject in the first ground truth image sequence during a testing period, the first ground truth image sequence comprising one or more drowsiness labels indicating a drowsiness of the test subject;
correlate a position on a circadian rhythm process to a time of the testing period;
apply one or more drowsiness corrections to the one or more drowsiness labels based at least on an alertness value corresponding to the position on the circadian rhythm process, to generate a second ground truth image sequence; and
train a machine learning model to infer a drowsiness level based on the second ground truth image sequence.
2 . The one or more processors of claim 1 , wherein the one or more processing units are further to:
determine the alertness value corresponding to the position on the circadian rhythm process based at least on a time of day and sensor data representing sleep information measured from the test subject.
3 . The one or more processors of claim 1 , wherein the one or more processing units are further to:
determine the alertness value corresponding to the position on the circadian rhythm process based at least on a time of day and sleep information based at least on responses to questions to the test subject.
4 . The one or more processors of claim 1 , wherein the one or more processing units are further to:
compute the one or more drowsiness corrections based at least on the alertness value corresponding to the position on the circadian rhythm process and time-on-task data associated with a task performed by the test subject during the testing period.
5 . The one or more processors of claim 1 , wherein the one or more processing units are further to:
compute the one or more drowsiness corrections based at least on the alertness value corresponding to the position on the circadian rhythm process and one or more alertness test measurements captured from the test subject during the testing period.
6 . The one or more processors of claim 5 , wherein the one or more alertness test measurements comprise at least one of an electroencephalogram (EEG) test and a mean reaction time (MRT) measurement.
7 . The one or more processors of claim 1 , wherein the first ground truth image sequence captures visual characteristics of the test subject performing one or more psychomotor vigilance tests (PVTs) over a course of the testing period.
8 . The one or more processors of claim 1 , wherein one or more drowsiness labels of the second ground truth image sequence comprise a score based at least on a Karolinska Sleepiness Scale (KSS).
9 . The one or more processors of claim 1 , wherein visual characteristics of the test subject in the first ground truth image sequence comprise at least one of an eye blink rate, an eye blink velocity, an eye blink amplitude, a time of eye closure, a head pose, an eye gaze direction, and a pattern of yawning behavior.
10 . The one or more processors of claim 1 , wherein the circadian rhythm process corresponds to a circadian rhythm process C curve.
11 . The one or more processors of claim 1 , wherein the one or more processors are comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for three-dimensional assets;
a system for performing deep learning operations;
a system for performing remote operations;
a system for performing real-time streaming;
a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content;
a system implemented using an edge device;
a system implemented using a robot;
a system for performing conversational AI operations;
a system implementing one or more language models;
a system implementing one or more large language models (LLMs);
a system for generating synthetic data;
a system for generating synthetic data using AI;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
12 . A system comprising:
one or more processing units to:
determine a drowsiness score estimate for a test subject during a testing period based at least on correlating a circadian rhythm process to the test subject at a time of the testing period;
apply one or more drowsiness corrections based at least on the drowsiness score estimate to a first ground truth image sequence representing visual characteristics of the test subject during the testing period to generate a second ground truth image sequence, the one or more drowsiness corrections correcting one or more drowsiness labels of the first ground truth image sequence; and
train a machine learning model using the second ground truth image sequence to infer a drowsiness level.
13 . The system of claim 12 , wherein the one or more processing units are further to:
determine the drowsiness score estimate based at least on a circadian rhythm process C curve.
14 . The system of claim 12 , wherein visual characteristics of the test subject in the first ground truth image sequence comprise at least one of an eye blink rate, an eye blink velocity, an eye blink amplitude, a time of eye closure, a head pose, an eye gaze direction, and a pattern of yawning behavior.
15 . The system of claim 12 , wherein the one or more processing units are further to:
determine the drowsiness score estimate based at least on an alertness value corresponding to a position on the circadian rhythm process, the position determined based at least on a time of day and sensor data representing sleep information measured from the test subject.
16 . The system of claim 12 , wherein the one or more processing units are further to:
determine the drowsiness score estimate based at least on an alertness value corresponding to a position on the circadian rhythm process, the position determined based at least on a time of day and sleep information based at least on responses to questions to the test subject.
17 . The system of claim 12 , wherein the one or more processing units are further to:
compute the one or more drowsiness corrections based at least on an alertness value corresponding to a position on the circadian rhythm process and time-on-task data associated with a task performed by the test subject during the testing period.
18 . The system of claim 12 , wherein the one or more processing units are further to:
compute the one or more drowsiness corrections based at least on an alertness value corresponding to a position on the circadian rhythm process and one or more alertness test measurements captured from the test subject during the testing period.
19 . The system of claim 12 , wherein the one or more processing units are comprised in at least one of:
a control system for an autonomous or semi-autonomous machine;
a perception system for an autonomous or semi-autonomous machine;
a system for performing simulation operations;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for performing collaborative content creation for three-dimensional assets;
a system for performing deep learning operations;
a system for performing remote operations;
a system for performing real-time streaming;
a system for generating or presenting one or more of augmented reality content, virtual reality content, or mixed reality content;
a system implemented using an edge device;
a system implemented using a robot;
a system for performing conversational AI operations;
a system implementing one or more language models;
a system implementing one or more large language models (LLMs);
a system for generating synthetic data;
a system for generating synthetic data using AI;
a system incorporating one or more virtual machines (VMs);
a system implemented at least partially in a data center; or
a system implemented at least partially using cloud computing resources.
20 . A method comprising:
generating a second ground truth image sequence from a first ground truth image sequence by applying one or more drowsiness corrections to one or more labels of the first ground truth image sequence, the one or more drowsiness corrections determined using a drowsiness score estimate for a test subject during a testing period associated with the first ground truth image sequence, the drowsiness score estimate based at least on correlating a circadian rhythm process to the test subject at a time of the testing period.