Systems and methods for extracting information from a dialogue
Described herein are systems and methods of extracting information from a dialogue, the dialogue having transcription data associated therewith. In an embodiment, the method including: receiving the transcription data associated with the dialogue; classifying utterances in the transcription data using a trained classification machine learning model, the classification machine learning model trained using one or more corpora of historical data comprising previous dialogues labelled with utterance types; identifying entities in the transcription data; classifying attributes in the transcription data using a trained attribute machine learning model, the attribute machine learning model trained using one or more corpora of historical data comprising previous dialogues labelled with attributes; and outputting at least one of the utterances, the entities, and the attributes.
1 . A computer-implemented method of extracting clinical information from textual data comprising a transcription of a patient-clinician dialogue, the method comprising:
receiving the textual data;
classifying utterances in the transcription data using a trained classification machine learning model, the classification machine learning model trained using one or more corpora of historical data comprising previous textual data labelled with utterances;
identifying entities in the transcription data;
classifying attributes in the transcription data using a trained attribute machine learning model, the attribute machine learning model trained using one or more corpora of historical data comprising previous textual data labelled with attributes;
generating a natural language clinical note, using a neural encoder-decoder model with copy and coverage mechanisms during the generating; and
outputting the natural language clinical note, the natural language clinical note comprising at least one of the utterances, the entities, and the attributes.
2 . The method of claim 1 , further comprising preprocessing the transcription data by one of stemming, lemmatization, part-of-speech tagging, and dependency parsing.
3 . The method of claim 1 , further comprising preprocessing the transcription data by tokenizing and removing stop-words and frequent-words.
4 . The method of claim 1 , wherein classifying the utterances comprising classifying as one of a question utterance, a statement utterance, a positive answer utterance, a negative answer utterance, a backchannel utterance, and an excluded utterance.
5 . The method of claim 1 , wherein the classification machine learning model comprises a two-layer bidirectional gated recurrent unit (GRU) neural network.
6 . The method of claim 5 , wherein each utterance can be represented as a multi-dimensional vector using a word embedding model.
7 . The method of claim 6 , wherein a first layer of the GRU network treats each utterance as a sequence of words and outputs a fixed-length utterance feature vector, and a second layer of the GRU network treats the dialogue as a sequence of the utterance feature vectors to generate a label for each utterance.
8 . The method of claim 1 , wherein identifying entities in the transcription data comprises identifying time expressions and converting the time expressions to standardized values using a temporal tagger.
9 . The method of claim 1 , wherein identifying entities in the transcription data comprises identifying medical concepts using comparison to a medical lexicon.
10 . The method of claim 1 , wherein the classified attributes comprise modality and pertinence, modality comprising an indication of whether an event associated with the attribute occurred, pertinence comprising an indication of the relevance of the attribute to a medical condition.
11 . The method of claim 1 , wherein identifying entities further comprises classifying each entity as one of subjective(S), objective (O), assessment (A), or plan (P).
12 . The method of claim 1 , further comprising classifying one or more diagnoses in the transcription data using a trained diagnoses machine learning model, and the output module further outputs the diagnoses.
13 . The method of claim 12 , further comprising identifying a primary diagnosis from the one or more diagnoses.
14 . The method of claim 1 , further comprising using topic modelling with an unsupervised model for extracting latent topics in the transcription of the dialogue.
15 . A system of extracting clinical information from textual data comprising a transcription of a patient-clinician dialogue, the system comprising one or more processors in communication with a data storage, the one or more processors configured to execute:
a data acquisition module to receive the textual data;
an utterance module to classify utterances in the transcription data using a trained classification machine learning model, the classification machine learning model trained using one or more corpora of historical data comprising previous textual data labelled with utterances;
an identifier module to identify entities in the transcription data;
an attribute module to classify attributes in the transcription data using a trained attribute machine learning model, the attribute machine learning model trained using one or more corpora of historical data comprising previous textual data labelled with attributes; and
an output module to generate a natural language clinical note using a neural encoder-decoder model with copy and coverage mechanisms, and to output the natural language clinical note, the natural language clinical note comprising at least one of the utterances, the entities, and the attributes.
16 . The system of claim 15 , further comprising a preprocessing module to preprocess the transcription data by one of stemming, lemmatization, part-of-speech tagging, and dependency parsing.
17 . The system of claim 15 , further comprising a preprocessing module to preprocess the transcription data by tokenizing and removing stop-words and frequent-words.
18 . The system of claim 15 , wherein classifying the utterances comprising classifying as one of a question utterance, a statement utterance, a positive answer utterance, a negative answer utterance, a backchannel utterance, and an excluded utterance.