Automated clinical documentation system and method for synchronizing machine vision and audio encounter information
A computer-implemented method, computer program product, and computing system for synchronizing machine vision and audio is executed on a computing device and includes obtaining encounter information of a user encounter, wherein the encounter information includes machine vision encounter information and audio encounter information. The machine vision encounter information and the audio encounter information are temporally-aligned to produce a temporarily-aligned encounter recording.
1 . A computer-implemented method comprising:
recording, by one or more machine vision devices of an automated clinical documentation (ACD) system, machine vision encounter information of a clinical encounter that includes a patient meeting with a participant in a clinical environment monitored by the ACD system, the one or more machine vision devices of the ACD system being deployed in the clinical environment;
recording, by one or more audio sensors of the ACD system, audio encounter information of the clinical encounter, the one or more audio sensors of the ACD system being deployed in the clinical environment, wherein the one or more audio sensors are not synchronized with the one or more machine vision devices such that the machine vision encounter information and the audio encounter information lack synchronization at the time of recording;
temporally-aligning the machine vision encounter information and the audio encounter information to produce a temporally-aligned encounter recording that synchronizes the machine vision encounter information with the audio encounter information;
generating, by the ACD system, a transcript of the clinical encounter based on the temporally-aligned encounter recording, wherein at least a portion of the transcript is generated by performing visual speech recognition on the synchronized machine vision encounter information;
proactively scanning, by the ACD system, the transcript based on a medical conditions symptoms datasource to identify indications of a specific medical condition, wherein the medical conditions symptoms datasource defines symptoms for various medical conditions;
sending, by the ACD system during the clinical encounter, a message to a participant of the clinical encounter responsive to determining that a number of the indications of the specific medical condition exceeds a threshold, the message recommending that the patient undergo testing for the specific medical condition; and
displaying, via a display of the ACD system, a visual representation of the transcript, the visual representation of the transcript identifying portions of the transcript that indicate the specific medical condition.
2 . The computer-implemented method of claim 1 , wherein the one or more machine vision devices include at least one of:
an RGB imaging system;
an infrared imaging system;
an ultraviolet imaging system;
a laser imaging system;
an X-ray imaging system;
a SONAR imaging system;
a RADAR imaging system; or
a thermal imaging system.
3 . The computer-implemented method of claim 1 , wherein the message recommending that the patient undergo testing for the specific medical condition is a text message sent to an electronic device of the participant.
4 . The computer-implemented method of claim 1 , wherein the message recommending that the patient undergo testing for the specific medical condition is an audio message.
5 . The computer-implemented method of claim 4 , wherein the audio message is played through an earbud worn by the participant during the clinical encounter.
6 . The computer-implemented method of claim 1 , wherein the message recommending that the patient undergo testing for the specific medical condition is displayed via the display of the ACD system.
7 . An automated clinical documentation (ACD) system comprising:
one or more machine vision devices configured to record machine vision encounter information of a clinical encounter that includes a patient meeting with a participant in a clinical environment monitored by the ACD system;
one or more audio sensors configured to record audio encounter information of the clinical encounter, the one or more audio sensors and the one or more machine vision devices of the ACD system being deployed in the clinical environment, wherein the one or more audio sensors are not synchronized with the one or more machine vision devices such that the machine vision encounter information and the audio encounter information lack synchronization at the time of recording, and wherein the machine vision encounter information and the audio encounter information are temporally aligned to produce a temporally-aligned encounter recording that synchronizes the machine vision encounter information with the audio encounter information;
at least one processor configured to generate a transcript of the clinical encounter based on the temporally-aligned encounter recording, wherein at least a portion of the transcript is generated by performing visual speech recognition on the synchronized machine vision encounter information, and wherein the at least one processor is further configured to proactively scan the transcript based on a medical conditions symptoms datasource to identify indications of a specific medical condition, wherein the medical conditions symptoms datasource defines symptoms for various medical conditions, and wherein the ACD system sends a message to a participant of the clinical encounter during the clinical encounter responsive to determining that a number of the indications of the specific medical condition exceeds a threshold, the message recommending that the patient undergo testing for the specific medical condition; and
a display configured to display a visual representation of the transcript, the visual representation of the transcript identifying portions of the transcript that indicate the specific medical condition.
8 . The ACD system of claim 7 , wherein the one or more machine vision devices include at least one of:
an RGB imaging system;
an infrared imaging system;
an ultraviolet imaging system;
a laser imaging system;
an X-ray imaging system;
a SONAR imaging system;
a RADAR imaging system; or
a thermal imaging system.
9 . The ACD system of claim 7 , wherein the message recommending that the patient undergo testing for the specific medical condition is a text message sent to an electronic device of the participant.
10 . The ACD system of claim 7 , wherein the message recommending that the patient undergo testing for the specific medical condition is an audio message.
11 . The ACD system of claim 10 , wherein the audio message is played through an earbud worn by the participant during the clinical encounter.
12 . The ACD system of claim 7 , wherein the message recommending that the patient undergo testing for the specific medical condition is displayed via the display of the ACD system.
13 . An automated clinical documentation (ACD) system comprising:
at least one processor; and
a memory storing programming instructions for execution by the at least one processor, the programming instructions, upon execution by the at least one processor, causing the ACD system to perform the following operations:
recording, by one or more machine vision devices of the ACD system, machine vision encounter information of a clinical encounter that includes a patient meeting with a participant in a clinical environment monitored by the ACD system, the one or more machine vision devices of the ACD system being deployed in the clinical environment;
recording, by one or more audio sensors of the ACD system, audio encounter information of the clinical encounter, the one or more audio sensors of the ACD system being deployed in the clinical environment, wherein the one or more audio sensors are not synchronized with the one or more machine vision devices such that the machine vision encounter information and the audio encounter information lack synchronization at the time of recording;
temporally-aligning the machine vision encounter information and the audio encounter information to produce a temporally-aligned encounter recording that synchronizes the machine vision encounter information with the audio encounter information;
generating, by the ACD system, a transcript of the clinical encounter based on the temporally-aligned encounter recording, wherein at least a portion of the transcript is generated by performing visual speech recognition on the synchronized machine vision encounter information;
proactively scanning, by the ACD system, the transcript based on a medical conditions symptoms datasource to identify indications of a specific medical condition, wherein the medical conditions symptoms datasource defines symptoms for various medical conditions;
sending, by the ACD system during the clinical encounter, a message to a participant of the clinical encounter responsive to determining that a number of the indications of the specific medical condition exceeds a threshold, the message recommending that the patient undergo testing for the specific medical condition; and
displaying, via a display of the ACD system, a visual representation of the transcript, the visual representation of the transcript identifying portions of the transcript that indicate the specific medical condition.
14 . The ACD system of claim 13 , wherein the one or more machine vision devices include at least one of:
an RGB imaging system;
an infrared imaging system;
an ultraviolet imaging system;
a laser imaging system;
an X-ray imaging system;
a SONAR imaging system;
a RADAR imaging system; or
a thermal imaging system.
15 . The ACD system of claim 13 , wherein the message recommending that the patient undergo testing for the specific medical condition is a text message sent to an electronic device of the participant.
16 . The ACD system of claim 13 , wherein the message recommending that the patient undergo testing for the specific medical condition is an audio message.
17 . The ACD system of claim 16 , wherein the audio message is played through an earbud worn by the participant during the clinical encounter.
18 . The ACD system of claim 13 , wherein the message recommending that the patient undergo testing for the specific medical condition is displayed via the display of the ACD system.