Systems and methods for natural language processing documentation review system for clinical decision support
A computer-implemented method for digitizing clinical documents comprises receiving a clinical document and a policy type. The method further comprises de-skewing and removing noise from the page. The method further comprises generating extracted text from the processed page using OCR, analyzing the extracted text to generate a classification label for the page using a multinomial classifier model comprising an LSTM neural network, which has been trained to recognize different classes of page based on similarities to a corpus of historical pages that have been previously classified. The method further comprises recognizing a specified keyword within the extracted text, using an NER that is trained using policy data relating to a policy having the policy type, and displaying a visual representation of the clinical document with a visual indicator of the presence of the specified keyword on the page, and a visual indicator of the classification label.
1 . A computer-implemented method for digitizing clinical documents, the method comprising:
receiving a clinical document and a policy type relating to the clinical document, the clinical document comprising at least one page;
generating a processed page, by
detecting whether the page includes an orientation skew;
de-skewing the page;
detecting whether the page includes noise; and
removing noise from the page;
generating extracted text from the processed page using optical character recognition (OCR);
analyzing the extracted text to generate a classification label for the page using a multinomial classifier model comprising a long short-term memory (LSTM) neural network, wherein the multinomial classifier model has been trained to recognize different classes of page, based on similarities to a corpus of historical pages that have been previously classified;
recognizing a specified keyword within the extracted text, using a named entity recognition model (NER), wherein the NER is trained using policy data relating to a policy having the policy type, and wherein the specified keyword is identified as relating to the policy type; and
displaying a visual representation of the clinical document on a display, wherein the visual representation comprises a visual indicator of the presence of the specified keyword on the page, and a visual indicator of the classification label.
2 . The method of claim 1 , further comprising correcting grammatical and textual abnormalities in the extracted text using a rules-based approach.
3 . The method of claim 1 , further comprising classifying an image on the page, using a convolutional neural network (CNN) architecture, wherein the CNN architecture is trained based on similarities to a corpus of historical pages that have been previously classified as comprising an image.
4 . The method of claim 3 , further comprising analyzing an image and text on the processed page to classify the processed page, using the CNN architecture to classify the image and the LSTM to classify the text.
5 . The method of claim 1 , wherein the de-skewing step and the removing noise step are performed using an OpenCV library.
6 . The method of claim 1 , further comprising the step of correcting a spelling of a word in the extracted text by choosing a corrected word, based at least in part on the frequency with which the corrected word appears in at least one of the corpus of historical pages and the policy data.
7 . The method of claim 1 , further comprising analyzing the extracted text to determine a location of a header, wherein the multinomial classifier model uses the location of the header to generate the classification label.
8 . The method of claim 1 , wherein the multinomial classifier model has been trained to recognize different classes of page based on the policy type in addition to the similarities to the corpus of historical pages that have been previously classified.
9 . The method of claim 1 , wherein the NER is trained using medical policy-specific keywords.
10 . A non-transitory computer-readable storage medium storing one or more programs for execution by one or more processors of an electronic device, the one or more programs comprising instructions for:
receiving a clinical document and a policy type relating to the clinical document, the clinical document comprising at least one page;
generating a processed page, by
detecting whether the page includes an orientation skew;
de-skewing the page;
detecting whether the page includes noise; and
removing noise from the page;
generating extracted text from the processed page using optical character recognition (OCR);
analyzing the extracted text to generate a classification label for the page using a multinomial classifier model comprising a long short-term memory (LSTM) neural network, wherein the multinomial classifier model has been trained to recognize different classes of page, based on similarities to a corpus of historical pages that have been previously classified;
recognizing a specified keyword within the extracted text, using a named entity recognition model (NER), wherein the NER is trained using policy data relating to a policy having the policy type, and wherein the specified keyword is identified as relating to the policy type; and
displaying a visual representation of the clinical document on a display, wherein the visual representation comprises a visual indicator of the presence of the specified keyword on the page and a visual indicator of the classification label.
11 . The medium of claim 10 , further comprising instructions for correcting grammatical and textual abnormalities in the extracted text using a rules-based approach.
12 . The medium of claim 10 , further comprising instructions for classifying an image on the page, using a convolutional neural network (CNN) architecture, wherein the CNN architecture is trained based on similarities to a corpus of historical pages that have been previously classified as comprising an image.
13 . The medium of claim 12 , further comprising instructions for analyzing an image and text on the processed page to classify the processed page, using the CNN architecture to classify the image and the LSTM to classify the text.
14 . The medium of claim 10 , further comprising instructions for correcting a spelling of a word in the extracted text by choosing a corrected word, based at least in part on the frequency with which the corrected word appears in at least one of the corpus of historical pages and the policy data.
15 . The medium of claim 10 , further comprising instructions for analyzing the extracted text to determine a location of a header, wherein the multinomial classifier model uses the location of the header to generate the classification label.
16 . A system for automating clinical documentation review, the system comprising: one or more processors;
memory; and
one or more programs stored in the memory, wherein the one or more programs are configured for execution by the one or more processors and include instructions for:
receiving a clinical document and a policy type relating to the clinical document, the clinical document comprising at least one page;
generating a processed page, by
detecting whether the page includes an orientation skew;
de-skewing the page;
detecting whether the page includes noise; and
removing noise from the page;
generating extracted text from the processed page using optical character recognition (OCR);
analyzing the extracted text to generate a classification label for the page using a multinomial classifier model comprising a long short-term memory (LSTM) neural network, wherein the multinomial classifier model has been trained to recognize different classes of page, based on similarities to a corpus of historical pages that have been previously classified;
recognizing a specified keyword within the extracted text, using a named entity recognition model (NER), wherein the NER is trained using policy data relating to a policy having the policy type, and wherein the specified keyword is identified as relating to the policy type; and
displaying a visual representation of the clinical document on a display, wherein the visual representation comprises a visual indicator of the presence of the specified keyword on the page, and a visual indicator of the classification label.
17 . The system of claim 16 , the one or more programs further comprising instructions for correcting grammatical and textual abnormalities in the extracted text using a rules-based approach.
18 . The system of claim 16 , the one or more programs further comprising instructions for classifying an image on the page, using a convolutional neural network (CNN) architecture, wherein the CNN architecture is trained based on similarities to a corpus of historical pages that have been previously classified as comprising an image.
19 . The system of claim 18 , the one or more programs further comprising instructions for analyzing an image and text on the processed page to classify the processed page, using the CNN architecture to classify the image and the LSTM to classify the text.
20 . The system of claim 16 , the one or more programs further comprising at least one of: (i) instructions for correcting a spelling of a word in the extracted text by choosing a corrected word, based at least in part on the frequency with which the corrected word appears in at least one of the corpus of historical pages and the policy data; or (ii) instructions for analyzing the extracted text to determine a location of a header, wherein the multinomial classifier model uses the location of the header to generate the classification label.