Systems and methods for machine learning from medical records
Systems and methods for machine learning of medical records are provided. The system can execute multiple machine learning models on the medical records in parallel using multi-threaded approach wherein each machine learning model executes using its own, dedicated computational thread in order to significantly speed up the time with which relevant information can be identified from documents by the system. The multi-threaded machine learning models can include, but are not limited to, sentence classification models, comorbidity models, ICD models, body parts models, prescription models, and provider name models. The system can also utilize combined convolutional neural networks and long short-term models (CNN+LSTMs) as well as ensemble machine learning models to categorize sentences in medical records. The system can also extract service provider, medical specializations, and dates of service information from medical records.
1 . A machine learning system for automatically extracting information from medical records, comprising:
a memory storing a plurality of medical records; and
a processor in communication with the memory, the processor programmed to perform the steps of:
retrieving the plurality of medical records from the memory;
retrieving at least one document having pages of text from the plurality of medical records;
processing the pages of text to clean data in the pages of text;
processing the pages of text to extract a date of service from the text using a pattern matching algorithm executed by the processor, the pattern matching algorithm searching for a plurality of surrounding words in the pages of text using at least one key word and identifying and extracting a date from the plurality of surrounding words;
processing the pages of text using a classifier model to identify a type of page for each page of text, the classifier model identifying and labeling each page with a label comprising one of a start page label, and end page label, and an other page label;
bundling the pages of text into a bundle using the labels such that bundled pages are considered to correspond to a visit by a patient to a medical provider;
for each page of each bundle, counting all dates that appear on the same page; and
assigning a date having a maximum count to each page of the bundle.
2 . The system of claim 1 , wherein the step of processing the pages of text to clean the data in the pages of text comprises removing e-mail addresses and links from the pages of text.
3 . The system of claim 2 , wherein the step of processing the pages of text to clean the data in the pages of text comprises removing non-English words and punctuation from the pages of text.
4 . The system of claim 3 , wherein the step of processing the pages of text to clean the data in the pages of text comprises removing stop words from the pages of text.
5 . The system of claim 4 , wherein the step of processing the pages of text to clean the data in the pages of text comprises removing small-length words from the pages of text.
6 . The system of claim 5 , wherein the step of processing the pages of text to clean the data in the pages of text comprises removing extra spaces and lower-case “the” letters from the pages of text.
7 . The system of claim 1 , wherein the step of processing the pages of text to extract the date of service from the text comprises extracting all dates in the page.
8 . A machine learning method for automatically extracting information from medical records, comprising:
retrieving a plurality of medical records from a memory;
retrieving at least one document having pages of text from the plurality of medical records;
processing the pages of text to clean data in the pages of text;
processing the pages of text to extract a date of service from the text using a pattern matching algorithm executed by the processor, the pattern matching algorithm searching for a plurality of surrounding words in the pages of text using at least one key word and identifying and extracting a date from the plurality of surrounding words;
processing the pages of text using a classifier model to identify a type of page for each page of text, the classifier model identifying and labeling each page with a label comprising one of a start page label, and end page label, and an other page label;
bundling the pages of text into a bundle using the labels such that bundled pages are considered to correspond to a visit by a patient to a medical provider;
for each page of each bundle, counting all dates that appear on the same page; and
assigning a date having a maximum count to each page of the bundle.
9 . The method of claim 8 , wherein the step of processing the pages of text to clean the data in the pages of text comprises removing e-mail addresses and links from the pages of text.
10 . The method of claim 9 , wherein the step of processing the pages of text to clean the data in the pages of text comprises removing non-English words and punctuation from the pages of text.
11 . The method of claim 10 , wherein the step of processing the pages of text to clean the data in the pages of text comprises removing stop words from the pages of text.
12 . The method of claim 11 , wherein the step of processing the pages of text to clean the data in the pages of text comprises removing small-length words from the pages of text.
13 . The method of claim 12 , wherein the step of processing the pages of text to clean the data in the pages of text comprises removing extra spaces and lower-case “the” letters from the pages of text.
14 . The method of claim 8 , wherein the step of processing the pages of text to extract the date of service from the text comprises extracting all dates in the page.