Computerized systems and methods for continuous and real-time fatigue detection based on computer vision analysis
According to some embodiments, disclosed are systems and methods for a novel framework that performs management of a location and the individuals operating therein based on determined fatigue data of such individuals. The framework may track a person (e.g., a user) at or around a location. Such tracking may be performed based on captured digital imagery of the user via a set of strategically positioned cameras at the location. In some embodiments, as soon as a user begins working, or upon detection by a camera(s), the framework may cause the camera(s) to begin capturing footage of the user, which may be fed, uploaded and/or streamed to a fatigue detection system that determines fatigue data related to the user. Such fatigue data may be leveraged to control which jobs certain users are performing, while reassigning other users based on safety decisions formed from their respective fatigue data.
1. A method comprising:
receiving, by at least one processor, a sequence of images from a digital imaging device;
wherein the sequence of images capture a sequence of positions in a movement performed by a user;
utilizing, by the at least one processor, at least one behavior determination machine learning model to generate a plurality of movement measurements for each image in the sequence of images based at least in part on:
the sequence of positions in the sequence of images, and
at least one behavior determination machine learning layer comprising a plurality of behavior determination machine learning parameters trained to ingest the sequence of images and output the plurality of movement measurements according to training on historical images;
determine, by the at least one processor, a plurality of movement features based at least in part on the plurality of movement measurements for each image in the sequence of images;
utilizing, by the at least one processor, at least one fatigue score machine learning model to generate at least one fatigue score based at least in part on:
the plurality of movement features, and
at least one fatigue score layer comprising a plurality of fatigue score parameters trained to ingest the plurality of movement features and output the at least one fatigue score according to training based on training movement measurements;
wherein the at least one fatigue score is indicative of a degree of fatigue exhibited in the sequence of positions of the movement performed by the user; and
updating, by the at least one processor, at least one activity log associated with the user based at least in part on at least one fatigue score.
2. The method of claim 1 , further comprising:
receiving, by the at least one processor, at least one fatigue self-scoring from at least one computing device associated with the user;
wherein the at least one fatigue self-scoring represents a user input defining a degree of fatigue associated with the movement; and
retraining, by the at least one processor, the plurality of fatigue score parameters based at least in part on an error between the at least one fatigue self-score and the at least one fatigue score.
3. The method of claim 1 , further comprising:
generating, by the at least one processor, a fatigue risk alert associated with the user based at least in part on the at least one fatigue score being below a predetermined fatigue threshold;
wherein the fatigue risk alert represents an increased risk of injury to the user due to fatigue; and
transmitting, by the at least one processor, the fatigue risk alert to at least one computing device so as to provide an alert of the fatigue risk, wherein the fatigue risk alert is configured to cause the at least one computing device to render at least one graphical user interface element indicative of the fatigue risk alert.
4. The method of claim 3 , wherein the at least one computing device is associated with at least one manager in a workplace of the user.
5. The method of claim 1 , further comprising:
accessing, by the at least one processor, the at least one activity log associated with the user; and
determining, by the at least one processor, at least one activity adjustment recommendation based at least in part on the at least one fatigue score, wherein the at least one activity adjustment recommendation indicates a change to the movement of the user to decrease a risk of injury due to fatigue.
6. The method of claim 5 , wherein the at least one activity adjustment recommendation comprises at least one decreased activity period indicative of a decrease in at least one movement-related attribute;
wherein the at least one movement-related attribute comprises at least one of:
a movement intensity, and
a movement frequency.
7. The method of claim 1 , further comprising:
determining, by the at least one processor, a user identifier associated with the user of the sequence of images based at least in part on at least one identifiable feature in at least one image of the sequence of images; and
identifying, by the at least one processor, the at least one activity log associated with the user based at least in part on the user identifier.
8. The method of claim 1 , further comprising:
generating, by the at least one processor, a plurality of sub-sequences of images from the sequence of images;
wherein each sub-sequence of images is associated with a window of time;
utilizing, by the at least one processor for each sub-sequence of images, the at least one behavior determination machine learning model to generate the plurality of movement measurements for each image;
determine, by the at least one processor for each sub-sequence of images, the plurality of movement features based at least in part on the plurality of movement measurements for each image;
utilizing, by the at least one processor for each sub-sequence of images, the at least one fatigue score machine learning model to generate the at least one fatigue score; and
updating, by the at least one processor for each sub-sequence of images, the at least one activity log associated with the user based at least in part on the at least one fatigue score for the window of time associated with each sub-sequence of images.
9. The method of claim 8 , further comprising:
determining, by the at least one processor, a statistical fatigue metric associated with a statistical aggregation of the at least one fatigue score for the window of time associated with each sub-sequence of images;
determining, by the at least one processor, that the at least one fatigue score for a particular window of time associated with a particular sub-sequence of images exceeds a predetermined threshold value;
generating, by the at least one processor, a fatigue risk alert associated with the user based at least in part on the at least one fatigue score being exceeding the predetermined threshold value;
wherein the fatigue risk alert represents an increased risk of injury to the user due to fatigue; and
transmitting, by the at least one processor, the fatigue risk alert to at least one computing device so as to provide an alert of the fatigue risk, wherein the fatigue risk alert is configured to cause the at least one computing device to render at least one graphical user interface element indicative of the fatigue risk alert.
10. The method of claim 9 , further comprising determining, by the at least one processor, at least one activity adjustment recommendation based at least in part on the statistical fatigue metric, wherein the at least one activity adjustment recommendation indicates a change to the movement of the user to decrease a risk of injury due to fatigue.
11. A system comprising:
at least one edge device comprising at least one processor, wherein the at least one processor is in communication with a non-transitory computer readable medium having software instructions stored thereon, wherein, upon execution of the software instructions, the at least one processor is configured to:
receive a sequence of images from a digital imaging device;
wherein the sequence of images capture a sequence of positions in a movement performed by a user;
utilize at least one behavior determination machine learning model to generate a plurality of movement measurements for each image in the sequence of images based at least in part on:
the sequence of positions in the sequence of images, and at least one behavior determination machine learning layer comprising a plurality of behavior determination machine learning parameters trained to ingest the sequence of images and output the plurality of movement measurements according to training on historical images;
determine, by the at least one processor, a plurality of movement features based at least in part on the plurality of movement measurements for each image in the sequence of images;
utilize at least one fatigue score machine learning model to generate at least one fatigue score based at least in part on:
the plurality of movement features, and
at least one fatigue score layer comprising a plurality of fatigue score regression parameters trained to ingest the plurality of movement features and output the at least one fatigue score according to training on movement measurements;
wherein the at least one fatigue score is indicative of a degree of fatigue exhibited in the sequence of positions of the movement performed by the user; and
update at least one activity log associated with the user based at least in part on at least one fatigue score.
12. The system of claim 11 , wherein, upon execution of the software instructions, the at least one processor is further configured to:
receive at least one fatigue self-scoring from at least one computing device associated with the user;
wherein the at least one fatigue self-scoring represents a user input defining a degree of fatigue associated with the movement; and
retrain the plurality of fatigue score parameters based at least in part on an error between the at least one fatigue self-score and the at least one fatigue score.
13. The system of claim 11 , wherein, upon execution of the software instructions, the at least one processor is further configured to:
generate a fatigue risk alert associated with the user based at least in part on the at least one fatigue score being below a predetermined fatigue threshold;
wherein the fatigue risk alert represents an increased risk of injury to the user due to fatigue; and
transmit the fatigue risk alert to at least one computing device so as to provide an alert of the fatigue risk, wherein the fatigue risk alert is configured to cause the at least one computing device to render at least one graphical user interface element indicative of the fatigue risk alert.
14. The system of claim 13 , wherein the at least one computing device is associated with at least one manager in a workplace of the user.
15. The system of claim 11 , wherein, upon execution of the software instructions, the at least one processor is further configured to:
access the at least one activity log associated with the user; and
determine at least one activity adjustment recommendation based at least in part on the at least one fatigue score, wherein the at least one activity adjustment recommendation indicates a change to the movement of the user to decrease a risk of injury due to fatigue.
16. The system of claim 15 , wherein the at least one activity adjustment recommendation comprises at least one decreased activity period indicative of a decrease in at least one movement-related attribute;
wherein the at least one movement-related attribute comprises at least one of:
a movement intensity, and
a movement frequency.
17. The system of claim 11 , wherein, upon execution of the software instructions, the at least one processor is further configured to:
determine a user identifier associated with the user of the sequence of images based at least in part on at least one identifiable feature in at least one image of the sequence of images; and
identify the at least one activity log associated with the user based at least in part on the user identifier.
18. The system of claim 11 , wherein, upon execution of the software instructions, the at least one processor is further configured to:
generate a plurality of sub-sequences of images from the sequence of images;
wherein each sub-sequence of images is associated with a window of time;
utilizing, by the at least one processor for each sub-sequence of images, the at least one behavior determination machine learning model to generate the plurality of movement measurements for each image;
determine, by the at least one processor for each sub-sequence of images, the plurality of movement features based at least in part on the plurality of movement measurements for each image;
utilizing, by the at least one processor for each sub-sequence of images, the at least one fatigue score machine learning model to generate the at least one fatigue score; and
updating, by the at least one processor for each sub-sequence of images, the at least one activity log associated with the user based at least in part on the at least one fatigue score for the window of time associated with each sub-sequence of images.
19. The system of claim 18 , wherein, upon execution of the software instructions, the at least one processor is further configured to:
determine a statistical fatigue metric associated with a statistical aggregation of the at least one fatigue score for the window of time associated with each sub-sequence of images;
determine that the at least one fatigue score for a particular window of time associated with a particular sub-sequence of images exceeds a predetermined threshold value;
generate a fatigue risk alert associated with the user based at least in part on the at least one fatigue score being exceeding the predetermined threshold value;
wherein the fatigue risk alert represents an increased risk of injury to the user due to fatigue; and
transmit the fatigue risk alert to at least one computing device so as to provide an alert of the fatigue risk, wherein the fatigue risk alert is configured to cause the at least one computing device to render at least one graphical user interface element indicative of the fatigue risk alert.
20. The system of claim 19 , wherein, upon execution of the software instructions, the at least one processor is further configured to determine at least one activity adjustment recommendation based at least in part on the statistical fatigue metric, wherein the at least one activity adjustment recommendation indicates a change to the movement of the user to decrease a risk of injury due to fatigue.