Radiology report editing method and system
The invention provides a radiology report editing method and system. The method comprises providing a radiology report; recording a speech command from a user to generate a speech file; processing the speech file using a speech recognition model that has been trained using a speech dataset comprising Vietnamese speeches labeled with ground truth text transcriptions to generate a text command; processing the text command using a natural language understanding model that has been trained to perform a classification task and a sequence tagging task, wherein the classification task classifies the text command into an intent, and wherein the sequence tagging task tags each word in the text command with a tagging sequence indicates whether the each word express an intent, a content or a position; extracting a content to be edited, a position of a sentence to be edited in the text command based on the output of the sequence tagging task; and editing the radiology report based on the extracted content, the extracted position, and the extracted intent of the text command.
1. A radiology report editing method:
providing a radiology report to be edited;
recording a speech command from a user to generate a speech file;
processing the speech file using a speech recognition model that has been trained using a speech dataset comprising Vietnamese speeches labeled with ground truth text transcriptions to generate a text command;
processing the text command using a natural language understanding model that has been trained to perform a classification task and a sequence tagging task;
wherein the classification task classifies the text command into an intent among a plurality of intents comprising an addition intent, a modification intent, a deletion intent and an unknown intent;
wherein the sequence tagging task tags each word in the text command with a tagging sequence, wherein the tagging sequence is selected from a group comprising an intent tagging sequence, a content tagging sequence and a position tagging sequence, wherein the intent tagging sequence indicates the classified intent of the text command; the content tagging sequence indicates a content to be edited and the position tagging sequence indicates a position of a sentence to be edited in the radiology report;
extracting the content and the position in the text command based on the content tagging sequence, the position tagging sequence, respectively;
if the classified intent is not the unknown intent, editing the radiology report based on the extracted content, the extracted position, and the classified intent of the text command;
wherein the natural language understanding model is trained using a training dataset,
wherein the training dataset includes a plurality of text commands that are generated based on a plurality of sentences extracted from a plurality of radiology reports,
wherein the plurality of text commands represents a plurality of intents comprising an addition intent, a modification intent, a deletion intent and an unknown intent;
wherein the plurality of text commands are generated using operations of:
providing a template that is selected from a group of an addition template, a modification template and a deletion template that represent an expression manner in Vietnamese for the addition intent, the modification intent or the deletion intent, respectively; and
filling data originated from the plurality of sentences into the template to create a first text command wherein the filled-in data comprise a content to be input and a position of a sentence in the radiology report to be edited.
2. The radiology report editing method of claim 1 ,
wherein the plurality of text commands are generated further using an operation of classifying each sentence in the plurality of sentences as a second text command representing the unknown intent.
3. The radiology report editing method of claim 2 , wherein the natural language understanding model comprises:
a BERT (Bidirectional Encoder Representations from Transformers)-based model that has been pre-trained for Vietnamese;
a softmax layer; and
a conditional random field (CRF) layer;
wherein the classification task and the sequence tagging task are jointly trained using the BERT-based model;
wherein the classification task is trained using the BERT-based model and the softmax layer; and
wherein the sequence tagging task is trained using the BERT-based model and the CRF layer.
4. The radiology report editing method of claim 3 , wherein the classification task and the sequence tagging task are jointly trained using the operations of:
tokenizing an input text command to generate a tokenized sequence of a plurality of tokens that comprises a plurality of word tokens, a classification [CLS] token and a separator [SEP] token; wherein the [CLS] token and the [SEP] token are added to the beginning and the end of the tokenized sequence, respectively;
processing the plurality of tokens using the BERT-based model to generate a [CLS] token representation that is an aggregate of entire input representation of the input command and a plurality of representations for the plurality of word tokens;
feeding the [CLS] token representation to the softmax layer to predict an intent of the input text command;
classifying the plurality of representations for the plurality of word tokens into a plurality of tags and feeding the plurality of tags to the CRF layer to obtain the plurality of tagging sequences;
minimizing a first loss measuring a difference between the predicted intent and a ground truth intent of the input text command; and
minimizing a second loss measuring a difference between the obtained plurality of tagging sequences and ground truth tagging sequences of the word tokens.
5. The radiology report editing method of claim 4 , wherein the recording of the speech command comprising transforming a time series audio signal stored in the speech file into a mel spectrogram.
6. The radiology report editing method of claim 5 , wherein the speech dataset for training the speech recognition model includes two parts, wherein the first part includes speeches in general language and the second part includes speeches in radiology.
7. The radiology report editing method of claim 6 , wherein the radiology report is in Vietnamese.
8. A radiology report editing system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one or more computers to perform operations comprising:
providing a radiology report to be edited;
recording a speech command from a user to generate a speech file;
processing the speech file using a speech recognition model that has been trained using a speech dataset comprising Vietnamese speeches labeled with ground truth text transcriptions to generate a text command;
processing the text command using a natural language understanding model that has been trained to perform a classification task and a sequence tagging task;
wherein the classification task classifies the text command into an intent among a plurality of intents comprising an addition intent, a modification intent, a deletion intent and an unknown intent;
wherein the sequence tagging task tags each word in the text command with a tagging sequence, wherein the tagging sequence is selected from a group comprising an intent tagging sequence, a content tagging sequence and a position tagging sequence, wherein the intent tagging sequence indicates the classified intent of the text command; the content tagging sequence indicates a content to be edited and the position tagging sequence indicates a position of a sentence to be edited in the radiology report;
extracting the content and the position in the text command based on the content tagging sequence, the position tagging sequence, respectively;
if the classified intent is not the unknown intent, editing the radiology report based on the extracted content, the extracted position, and the classified intent of the text command;
wherein the natural language understanding model is trained using a training dataset,
wherein the training dataset includes a plurality of text commands that are generated based on a plurality of sentences extracted from a plurality of radiology reports,
wherein the plurality of text commands represents a plurality of intents comprising an addition intent, a modification intent, a deletion intent and an unknown intent;
wherein the plurality of text commands are generated using operations of:
providing a template that is selected from a group of an addition template, a modification template and a deletion template that represent an expression manner in Vietnamese for the addition intent, the modification intent or the deletion intent, respectively; and
filling data originated from the plurality of sentences into the template to create a first text command wherein the filled-in data comprise a content to be input and a position of a sentence in the radiology report to be edited.