Systems and methods for detecting documentation drop-offs in clinical documentation
In clinical documentation, mere documentation of a condition in a patient's records may not be enough. To be considered sufficiently documented, the patient's record needs to show that no documentation drop-offs (DDOs) have occurred over the course of the patient's stay. However, DDOs can be extremely difficult to detect. To solve this problem, the invention trains time-sensitive deep learning (DL) models on a per condition basis using actual and/or synthetic patient data. Utilizing an ontology, grouped concepts can be generated on the fly from real-time hospital data and used to generate time-series data that can then be analyzed by trained time-sensitive DL models to determine whether a DDO for a condition has occurred during the stay. Non-time-sensitive models can be used to detect all the conditions documented during the stay. Outcomes from the models can be compared to determine whether to notify a user that a DDO has occurred.
1. A method, comprising:
responsive to new data arriving at a data store, processing, by a computer, entities of interest extracted from the new data into concepts specific to a patient;
automatically summarizing, by the computer, the concepts specific to the patient into concept groups, the concept groups indicative of medical conditions of the patient documented at time points during the patient's stay at a facility;
providing, by the computer, the concept groups to non-time-sensitive models and time-sensitive models;
wherein, on a per medical condition basis, a non-time-sensitive machine learning engine applies the non-time-sensitive models to the concept groups across all the medical conditions of the patient and generates an output containing a list of documented medical conditions that the patient likely has during the patient's stay; and
wherein, on a per medical condition basis, a time-sensitive machine learning engine applies the time-sensitive models to the concept groups across the medical conditions of the patient documented within a time period and generates an output containing a running total of documented medical conditions that the patient likely has sufficiently documented during the time period;
comparing, by the computer, the output from the non-time-sensitive machine learning engine and the output from the time-sensitive machine learning engine to identify any medical condition of the patient that is identified by the non-time-sensitive machine learning engine as having been documented during the patient's stay, but that is not identified by the time-sensitive machine learning engine as having been sufficiently documented during the time period, indicating that a documentation drop-off (DDO) has occurred; and
generating, by the computer, a notification that the DDO has occurred.
2. The method according to claim 1 , wherein the processing comprises:
parsing the entities of interest from the new data; and
categorizing, in real time utilizing a dictionary, the entities of interest into the concepts specific to the patient, wherein the dictionary contains predefined concepts and texts representing the predefined concepts.
3. The method according to claim 1 , further comprising:
providing timestamps associated with the concept groups to the time-sensitive machine learning engine; and
generating, utilizing the timestamps, an input time-series with a binary answer or indicator for each of the concept groups, indicating a temporal pattern of the concept groups documented at the time points during the patient's stay at the facility.
4. The method according to claim 3 , further comprising:
determining, based on historical time-series that are similar to the input time-series, a probability that a target medical condition is sufficiently documented during the patient's stay at the facility.
5. The method according to claim 4 , further comprising:
updating a patient record of the patient to include the probability that the target medical condition is sufficiently documented during the patient's stay at the facility; and
fine-tuning the time-sensitive machine learning engine using the patient record thus updated so as to improve performance of the time-sensitive machine learning engine.
6. The method according to claim 5 , further comprising:
repeating the updating and the fine-tuning for each of the medical conditions of the patient.
7. The method according to claim 5 , wherein the patient record includes semantic attributes about each concept specific to the patient.
8. A system, comprising:
a processor;
a non-transitory computer-readable medium; and
instructions stored on the non-transitory computer-readable medium and translatable by the processor for:
responsive to new data arriving at a data store, processing entities of interest extracted from the new data into concepts specific to a patient;
automatically summarizing the concepts specific to the patient into concept groups, the concept groups indicative of medical conditions of the patient documented at time points during the patient's stay at a facility;
providing the concept groups to non-time-sensitive models and time-sensitive models;
wherein, on a per medical condition basis, a non-time-sensitive machine learning engine applies the non-time-sensitive models to the concept groups across all the medical conditions of the patient and generates an output containing a list of documented medical conditions that the patient likely has during the patient's stay; and
wherein, on a per medical condition basis, a time-sensitive machine learning engine applies the time-sensitive models to the concept groups across the medical conditions of the patient documented within a time period and generates an output containing a running total of documented medical conditions that the patient likely has sufficiently documented during the time period;
comparing the output from the non-time-sensitive machine learning engine and the output from the time-sensitive machine learning engine to identify any medical condition of the patient that is identified by the non-time-sensitive machine learning engine as having been documented during the patient's stay, but that is not identified by the time-sensitive machine learning engine as having been sufficiently documented during the time period, indicating that a documentation drop-off (DDO) has occurred; and
generating a notification that the DDO has occurred.
9. The system of claim 8 , wherein the processing comprises:
parsing the entities of interest from the new data; and
categorizing, in real time utilizing a dictionary, the entities of interest into the concepts specific to the patient, wherein the dictionary contains predefined concepts and texts representing the predefined concepts.
10. The system of claim 8 , wherein the instructions are further translatable by the processor for:
providing timestamps associated with the concept groups to the time-sensitive machine learning engine; and
generating, utilizing the timestamps, an input time-series with a binary answer or indicator for each of the concept groups, indicating a temporal pattern of the concept groups documented at the time points during the patient's stay at the facility.
11. The system of claim 10 , wherein the instructions are further translatable by the processor for:
determining, based on historical time-series that are similar to the input time-series, a probability that a target medical condition is sufficiently documented during the patient's stay at the facility.
12. The system of claim 11 , wherein the instructions are further translatable by the processor for:
updating a patient record of the patient to include the probability that the target medical condition is sufficiently documented during the patient's stay at the facility; and
fine-tuning the time-sensitive machine learning engine using the patient record thus updated so as to improve performance of the time-sensitive machine learning engine.
13. The system of claim 12 , wherein the instructions are further translatable by the processor for:
repeating the updating and the fine-tuning for each of the medical conditions of the patient.
14. The system of claim 12 , wherein the patient record includes semantic attributes about each concept specific to the patient.
15. A computer program product comprising a non-transitory computer-readable medium storing instructions translatable by a processor for:
responsive to new data arriving at a data store, processing entities of interest extracted from the new data into concepts specific to a patient;
automatically summarizing the concepts specific to the patient into concept groups, the concept groups indicative of medical conditions of the patient documented at time points during the patient's stay at a facility;
providing the concept groups to non-time-sensitive models and time-sensitive models;
wherein, on a per medical condition basis, a non-time-sensitive machine learning engine applies the non-time-sensitive models to the concept groups across all the medical conditions of the patient and generates an output containing a list of documented medical conditions that the patient likely has during the patient's stay; and
wherein, on a per medical condition basis, a time-sensitive machine learning engine applies the time-sensitive models to the concept groups across the medical conditions of the patient documented within a time period and generates an output containing a running total of documented medical conditions that the patient likely has sufficiently documented during the time period;
comparing the output from the non-time-sensitive machine learning engine and the output from the time-sensitive machine learning engine to identify any medical condition of the patient that is identified by the non-time-sensitive machine learning engine as having been documented during the patient's stay, but that is not identified by the time-sensitive machine learning engine as having been sufficiently documented during the time period, indicating that a documentation drop-off (DDO) has occurred; and
generating a notification that the DDO has occurred.
16. The computer program product of claim 15 , wherein the processing comprises:
parsing the entities of interest from the new data; and
categorizing, in real time utilizing a dictionary, the entities of interest into the concepts specific to the patient, wherein the dictionary contains predefined concepts and texts representing the predefined concepts.
17. The computer program product of claim 15 , wherein the instructions are further translatable by the processor for:
providing timestamps associated with the concept groups to the time-sensitive machine learning engine; and
generating, utilizing the timestamps, an input time-series with a binary answer or indicator for each of the concept groups, indicating a temporal pattern of the concept groups documented at the time points during the patient's stay at the facility.
18. The computer program product of claim 17 , wherein the instructions are further translatable by the processor for:
determining, based on historical time-series that are similar to the input time-series, a probability that a target medical condition is sufficiently documented during the patient's stay at the facility.
19. The computer program product of claim 18 , wherein the instructions are further translatable by the processor for:
updating a patient record of the patient to include the probability that the target medical condition is sufficiently documented during the patient's stay at the facility; and
fine-tuning the time-sensitive machine learning engine using the patient record thus updated so as to improve performance of the time-sensitive machine learning engine.
20. The computer program product of claim 19 , wherein the instructions are further translatable by the processor for:
repeating the updating and the fine-tuning for each of the medical conditions of the patient.