System and method for inverse summarization of medical records with data augmentation and knowledge distillation
A method, computer program product, and computing system for generating a first synthetic dataset including a synthetic transcription and a corresponding natural dictation record using a first machine learning model trained to generate transcriptions from medical records. A second synthetic dataset including a synthetic medical record and a corresponding natural transcription is generated using a second machine learning model trained to generate medical records from transcriptions. The first synthetic dataset and the second synthetic dataset are combined with a natural dataset into a synthetic training dataset.
1. A computer-implemented method, executed on a computing device, comprising:
generating a first synthetic dataset including a synthetic transcription and a corresponding natural dictation record using a first machine learning model trained to generate transcriptions from medical records;
generating a second synthetic dataset including a synthetic medical record and a corresponding natural transcription using a second machine learning model trained to generate medical records from transcriptions;
combining the first synthetic dataset and the second synthetic dataset with a natural dataset into a synthetic training dataset;
training a third machine learning model to generate a medical record from a transcription using the synthetic training dataset by knowledge distillation from the first machine learning model and the second machine learning model to the third machine learning model for generating the synthetic medical record from the transcriptions, wherein the third machine learning model is smaller than the first machine learning model and the second machine learning model; and
generating a medical record from a transcription using the third machine learning model.
2. The computer-implemented method of claim 1 , wherein generating the first synthetic dataset includes:
training the first machine learning model to generate transcriptions from medical records using a plurality of transcriptions and a plurality of corresponding medical records.
3. The computer-implemented method of claim 1 , wherein generating the first synthetic dataset includes:
generating the synthetic transcription using the corresponding natural dictation record during decoding.
4. The computer-implemented method of claim 3 , wherein generating the synthetic transcription using the corresponding natural dictation record during decoding includes:
weighting each output token of the synthetic transcription based upon, at least in part, the distance between a current output token position and a previous output token position and a number of instances of the output token being generated within the synthetic transcription.
5. The computer-implemented method of claim 1 , wherein generating the second synthetic dataset includes:
training the second machine learning model for generating medical records from transcriptions using a plurality of transcriptions and a plurality of corresponding medical records.
6. The computer-implemented method of claim 1 , wherein combining the first synthetic dataset and the second synthetic dataset with the natural dataset into the synthetic training dataset includes:
filtering the first synthetic dataset and the second synthetic dataset based upon, at least in part, a filtering rule set.
7. A computing system comprising:
a memory; and
a processor to perform operations comprising:
generating a first synthetic dataset including a synthetic transcription and a corresponding natural dictation record using a first machine learning model trained to generate transcriptions from medical records;
generating a second synthetic dataset including a synthetic medical record and a corresponding natural transcription using a second machine learning model trained to generate medical records from transcriptions;
combining the first synthetic dataset and the second synthetic dataset with a natural dataset into a synthetic training dataset, and;
training a third machine learning model to generate medical records using the synthetic training dataset by knowledge distillation from the first machine learning model and the second machine learning model to the third machine learning model for generating the synthetic medical record from the transcriptions, wherein the third machine learning model is smaller than the first machine learning model and the second machine learning model; and
generating a medical record from a transcription using the third machine learning model.
8. The computing system of claim 7 , wherein generating the first synthetic dataset includes:
training the first machine learning model to generate transcriptions from medical records using a plurality of transcriptions and a plurality of corresponding medical records.
9. The computing system of claim 7 , wherein generating the first synthetic dataset includes:
generating the synthetic transcription using the corresponding natural dictation record during decoding.
10. The computing system of claim 9 , wherein generating the synthetic transcription using the corresponding natural dictation record during decoding includes:
weighting each output token of the synthetic transcription based upon, at least in part, the distance between a current output token position and a previous output token position and a number of instances of the output token being generated within the synthetic transcription.
11. The computing system of claim 7 , wherein generating the second synthetic dataset includes:
training the second machine learning model for generating medical records from transcriptions using a plurality of transcriptions and a plurality of corresponding medical records.
12. The computing system of claim 7 , wherein generating the synthetic medical record includes:
generating the synthetic medical record using the corresponding natural transcription during decoding.
13. A computer program product residing on a non-transitory computer readable medium having a plurality of instructions stored there on which, when executed by a processor, cause the processor to perform operations comprising:
generating a first synthetic dataset including a synthetic transcription and a corresponding natural dictation record during decoding using a first machine learning model trained to generate transcriptions from medical records, wherein generating the first synthetic dataset during decoding includes weighting each output token of the synthetic transcription based upon, at least in part, the distance between a current output token position and a previous output token position and a number of instances of the output token to be generated within the synthetic transcription;
generating a second synthetic dataset including a synthetic medical record and a corresponding natural transcription using a second machine learning model trained to generate medical records from transcriptions;
combining the first synthetic dataset and the second synthetic dataset with a natural dataset into a synthetic training dataset;
training a third machine learning model to generate a medical record from a transcription using the synthetic training dataset by knowledge distillation from the first machine learning model and the second machine learning model to the third machine learning model for generating the synthetic medical record from the transcriptions, wherein the third machine learning model is smaller than the first machine learning model and the second machine learning model; and
generating a medical record from a transcription using the third machine learning model.
14. The computer program product of claim 13 , wherein generating the first synthetic dataset includes:
training the first machine learning model to generate transcriptions from medical records using a plurality of transcriptions and a plurality of corresponding medical records.
15. The computer program product of claim 13 , wherein weighting each output token of the synthetic transcription includes:
weighting each output token of the synthetic transcription as a ratio of the distance between a current output token position and a previous output token position and the number of instances of the output token to be generated within the synthetic transcription.
16. The computer program product of claim 13 , wherein generating the second synthetic dataset includes:
training the second machine learning model for generating medical records from transcriptions using a plurality of transcriptions and a plurality of corresponding medical records.
17. The computer program product of claim 13 , wherein combining the first synthetic dataset and the second synthetic dataset with the natural dataset into the synthetic training dataset includes:
filtering the first synthetic dataset and the second synthetic dataset based upon, at least in part, a filtering rule set.