Artificial intelligence for evaluating patient distress using facial emotion recognition
Artificial intelligence for evaluating patient distress using facial emotion recognition. In an embodiment, an artificial intelligence model is applied to facial image(s) of a patient to classify each facial image into one of a plurality of emotional states based on a facial expression in the facial image. A determination may be made as to whether or not to alert a healthcare provider based on the emotional state(s) into which the facial image(s) were classified. If a determination is made to alert a healthcare provider, a notification may be transmitted to the healthcare provider.
1 . A method for monitoring patients in a medical facility comprising using at least one hardware processor to, for each of one or more patients in the medical facility:
receive a plurality of facial images of the patient over a period of time, wherein each of the plurality of facial images includes a face of the patient;
generate a history of emotional states for the patient by, for each of the plurality of facial images, providing a facial image of the plurality of facial images as input into a first artificial-intelligence model, wherein the first artificial-intelligence model generates an output comprising a classification of the facial image into one of a plurality of emotional states based on a facial expression in the facial image, wherein the plurality of emotional states comprise a plurality of emotions;
evaluate the history of emotional states by providing the history of emotional states as input into a second artificial-intelligence model, wherein the second artificial-intelligence model generates an output indicative of an emotional state of the patient, wherein the second artificial-intelligence model is different from the first artificial-intelligence model;
based upon the output of the second artificial-intelligence model, determine whether or not to alert one or more recipients; and
upon determining to alert the one or more recipients, transmit a notification to each of the one or more recipients.
2 . The method of claim 1 , wherein the first artificial-intelligence model comprises a machine-learning model trained on a plurality of facial images that have each been labeled with one of the plurality of emotional states.
3 . The method of claim 2 , wherein the machine-learning model comprises a convolutional neural network.
4 . The method of claim 1 , further comprising using the at least one hardware processor to store the history of emotional states in memory.
5 . The method of claim 4 , wherein the one or more patients are a plurality of patients, and wherein the method further comprises using the at least one hardware processor to analyze the histories of emotional states, stored in the memory, across all of the plurality of patients, over time, to calculate an overall performance rating for the medical facility.
6 . The method of claim 1 , further comprising using a camera to capture each of the plurality of facial images, wherein the determination of whether or not to alert one or more recipients is made in real time with the capture of the plurality of facial images.
7 . The method of claim 1 , wherein at least one of the plurality of emotional states indicates satisfaction, and at least one of the plurality of emotional states indicates dissatisfaction.
8 . The method of claim 1 , wherein at least one of the plurality of emotional states indicates distress.
9 . The method of claim 8 , wherein at least one of the plurality of emotional states indicates pain.
10 . The method of claim 1 , wherein at least one of the plurality of emotional states indicates a risk of medical deterioration.
11 . The method of claim 1 , wherein the second artificial-intelligence model comprises a linear regression.
12 . The method of claim 1 , wherein each of the one or more recipients is a healthcare provider.
13 . The method of claim 12 , further comprising, for each of the one or more patients, capturing the plurality of facial images of the patient via a surveillance system in the medical facility.
14 . The method of claim 13 , wherein the surveillance system comprises a camera directed at a hospital bed.
15 . The method of claim 1 , wherein the output of the second artificial-intelligence model comprises a threshold percentage of an emotional state of the plurality of emotional states indicative of emotional distress.
16 . A system comprising:
at least one hardware processor; and
one or more software modules that are configured to, when executed by the at least one hardware processor, for each of one or more persons,
receive a plurality of facial images of the patient over a period of time, wherein each of the plurality of facial images includes a face of the person;
generate a history of emotional states for the patient by, for each of the plurality of facial images, providing a facial image of the plurality of facial images as input into a first artificial-intelligence model, wherein the first artificial-intelligence model generates an output comprising a classification of the facial image into one of a plurality of emotional states based on a facial expression in the facial image, wherein the plurality of emotional states comprise a plurality of emotions;
evaluate the history of emotional states by providing the history of emotional states as input into a second artificial-intelligence model, wherein the second artificial-intelligence model generates an output indicative of an emotional state of the patient, wherein the second artificial-intelligence model is different from the first artificial-intelligence model;
based upon the output of the second artificial-intelligence model, determine whether or not to alert one or more recipients; and
upon determining to alert the one or more recipients, transmit a notification to each of the one or more recipients.
17 . The system of claim 16 , wherein at least one of the plurality of emotional states indicates a risk of medical deterioration.
18 . The system of claim 16 , further comprising using a camera to capture each of the plurality of facial images, wherein the determination of whether or not to alert one or more recipients is made in real time with the capture of the plurality of facial images.
19 . The system of claim 18 , further comprising, for each of the one or more patients, capturing the plurality of facial images of the patient via a surveillance system in the medical facility.
20 . A non-transitory computer-readable medium having instructions stored therein, wherein the instructions, when executed by a processor, cause the processor to, for each of one or more persons:
receive a plurality of facial images of the patient over a period of time, wherein each of the plurality of facial images includes a face of the person;
generate a history of emotional states for the patient by, for each of the plurality of facial images, providing a facial image of the plurality of facial images as input into a first artificial-intelligence model, wherein the first artificial-intelligence model generates an output comprising a classification of the facial image into one of a plurality of emotional states based on a facial expression in the facial image, wherein the plurality of emotional states comprise a plurality of emotions;
evaluate the history of emotional states by providing the history of emotional states as input into a second artificial-intelligence model, wherein the second artificial-intelligence model generates an output indicative of an emotional state of the patient, wherein the second artificial-intelligence model is different from the first artificial-intelligence model;
based upon the output of the second artificial-intelligence model, determine whether or not to alert one or more recipients; and
upon determining to alert the one or more recipients, transmit a notification to each of the one or more recipients.