Automatic disease detection from unstructured textual reports

1. A method for detection of disease indications from textual descriptions and medical imagery, the method comprising:

reading a knowledge graph of clinical concepts;

based on the knowledge graph, determining a plurality of associations, each association comprising a tuple of a disease name, a symptom, an anatomical abnormality, and a qualifier;

reading a corpus of multiple unstructured textual clinical reports of a plurality of patients;

for each of the plurality of associations, searching the corpus of clinical reports for the respective tuple;

based on said searching, locating within the corpus of clinical reports a plurality of portions of the clinical reports, each of the plurality of portions containing one of the respective tuples, and each of the plurality of portions being indicative of a disease condition of a patient;

within each of the plurality of portions of the clinical reports, detecting name/value pairs corresponding to measurements indicative of the disease condition;

extracting from the portions of the clinical reports the measurements indicative of the disease condition;

extracting a plurality of features from a plurality of imaging studies of the plurality of patients using a first fully-connected layer of a convolutional neural network (CNN);

providing the plurality of features and the measurements to a trained classifier, the trained classifier comprising a support vector machine (SVM), the trained classifier being trained to output a disease label of a plurality of disease labels based on a plurality of ground truth images, wherein the training comprises:

providing a feature vector for each of the plurality of ground truth images to the SVM, the plurality of ground truth images comprising a plurality of ultrasound mode labels, each of the ground truth images comprising a valve label and an ultrasound mode label;

applying the SVM to the feature vectors of the ground truth images to thereby learn associations between the plurality of disease labels, the valve labels, and the mode labels;

obtaining from the trained classifier a disease label indicative of the disease condition based on the measurements and the plurality of features.

2. The method of claim 1 , wherein the knowledge graph is based on at least a phrase vocabulary.

3. The method of claim 1 , wherein each of the plurality of portions is a phrase.

4. The method of claim 1 , wherein the clinical reports comprise echocardiogram reports.

5. The method of claim 1 , further comprising:

generating the knowledge graph of clinical concepts from a plurality of reference vocabularies.

6. The method of claim 1 , wherein detecting name/value pairs comprises applying a search string to the plurality of portions of the clinical reports.

7. The method of claim 6 , wherein the search string comprise a regular expression.

8. The method of claim 1 , further comprising:

reading a plurality of measurement names;

generating a plurality of variant measurement names of the plurality of measurement names;

reading a corpus of sentences containing measurement names and values;

performing part of speech tagging of each sentence of the corpus of sentences;

based on the part of speech tagging, identifying measurement names associated with measurement values;

based on the identified measurement names and the plurality of variant measurement names, generating a plurality of patterns indicative of a following value, and wherein detecting name/value pairs comprises searching for the plurality of patterns.

9. A computer program product for detection of disease indications from textual descriptions and medical imagery, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform a method comprising:

reading a knowledge graph of clinical concepts;

based on the knowledge graph, determining a plurality of associations, each association comprising a tuple of a disease name, a symptom, an anatomical abnormality, and a qualifier;

reading a corpus of multiple unstructured textual clinical reports of a plurality of patients;

for each of the plurality of associations, searching the corpus of clinical reports for the respective tuples;

based on said searching, locating within the corpus of clinical reports a plurality of portions of the clinical reports, each of the plurality of portions containing one of the respective tuples, and each of the plurality of portions being indicative of a disease condition of a patient;

within each of the plurality of portions of the clinical reports, detecting name/value pairs corresponding to measurements indicative of the disease condition;

extracting from the portions of the clinical reports the measurements indicative of the disease condition;

extracting a plurality of features from a plurality of imaging studies of the plurality of patients using a first fully-connected layer of a convolutional neural network (CNN);

providing the plurality of features and the measurements to a trained classifier, the trained classifier comprising a support vector machine (SVM), the trained classifier being trained to output a disease label of a plurality of disease labels based on a plurality of ground truth images, wherein the training comprises:

providing a feature vector for each of the plurality of ground truth images to the SVM, the plurality of ground truth images comprising a plurality of ultrasound mode labels, each of the ground truth images comprising a valve label and an ultrasound mode label;

applying the SVM to the feature vectors of the ground truth images to thereby learn associations between the plurality of disease labels, the valve labels, and the mode labels;

obtaining from the trained classifier a disease label indicative of the disease condition based on the measurements and the plurality of features.

10. The computer program product of claim 9 , wherein the knowledge graph is based on at least a phrase vocabulary.

11. The computer program product of claim 9 , wherein each of the sequences is a phrase.

12. The computer program product of claim 9 , wherein the clinical reports comprise echocardiogram reports.

13. The computer program product of claim 9 , the method further comprising:

generating the knowledge graph of clinical concepts from a plurality of reference vocabularies.

14. The computer program product of claim 9 , wherein detecting name/value pairs comprises applying a search string to the plurality of portions of the clinical reports.

15. The computer program product of claim 14 , wherein the search string comprise a regular expression.

16. The computer program product of claim 9 , the method further comprising:

reading a plurality of measurement names;

generating a plurality of variant measurement names of the plurality of measurement names;

reading a corpus of sentences containing measurement names and values;

performing part of speech tagging of each sentence of the corpus of sentences;

based on the part of speech tagging, identifying measurement names associated with measurement values;

based on the identified measurement names and the plurality of variant measurement names, generating a plurality of patterns indicative of a following value, and wherein detecting name/value pairs comprises searching for the plurality of patterns.

17. A system comprising:

a first storage device comprising a knowledge graph of clinical concepts;

a second storage device comprising a corpus of clinical reports and medical imagery;

a computing node comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor of the computing node to cause the processor to perform a method for detection of disease indications from textual descriptions and medical imagery, the method comprising:

reading a knowledge graph of clinical concepts;

based on the knowledge graph, determining a plurality of associations, each association comprising a tuple of a disease name, a symptom, an anatomical abnormality, and a qualifier;

reading a corpus of multiple unstructured textual clinical reports of a plurality of patients;

for each of the plurality of associations, searching the corpus of clinical reports for the respective tuple;

based on said searching, locating within the corpus of clinical reports a plurality of portions of the clinical reports, each of the plurality of portions containing one of the respective tuples, and each of the plurality of portions being indicative of a disease condition of a patient;

within each of the plurality of portions of the clinical reports, detecting name/value pairs corresponding to measurements indicative of the disease condition;

extracting from the portions of the clinical reports the measurements indicative of the disease condition;

extracting a plurality of features from a plurality of imaging studies of the plurality of patients using a first fully-connected layer of a convolutional neural network (CNN);

providing the plurality of features and the measurements to trained classifier, the trained classifier comprising a support vector machine (SVM), the trained classifier being trained to output a disease label of a plurality of disease labels based on a plurality of ground truth images, wherein the training comprises:

providing a feature vector for each of the plurality of ground truth images to the SVM, the plurality of ground truth images comprising a plurality of ultrasound mode labels, each of the ground truth images comprising a valve label and an ultrasound mode label;

applying the SVM to the feature vectors of the ground truth images to thereby learn associations between the plurality of disease labels, the valve labels, and the mode labels;

obtaining from the trained classifier a disease label indicative of the disease condition based on the measurements and the plurality of features.

18. The system of claim 17 , wherein the clinical reports comprise echocardiogram reports.

19. The system of claim 17 , the method further comprising:

generating the knowledge graph of clinical concepts from a plurality of reference vocabularies.

20. The system of claim 17 , the method further comprising:

reading a plurality of measurement names;

generating a plurality of variant measurement names of the plurality of measurement names;

reading a corpus of sentences containing measurement names and values;

performing part of speech tagging of each sentence of the corpus of sentences;

based on the part of speech tagging, identifying measurement names associated with measurement values;

based on the identified measurement names and the plurality of variant measurement names, generating a plurality of patterns indicative of a following value, and wherein detecting name/value pairs comprises searching for the plurality of patterns.

21. The method of claim 1 , wherein extracting the plurality of features from the plurality of imaging studies comprises:

reading a plurality of frames from each of the plurality of imaging studies;

providing the plurality of frames to a trained feature generator;

obtaining from the trained feature generator the plurality of features corresponding to the plurality of frames.

22. The method of claim 1 , wherein extracting the plurality of features from the plurality of imaging studies comprises:

selecting a frame from a medical video of the plurality of imaging studies, the selected frame depicting a valve of interest;

extracting a Doppler envelope from the selected frame;

based on the frame and the Doppler envelope, extracting one or more measurements indicative of the disease condition.

23. The method of claim 22 , wherein selecting the frame comprises:

providing a plurality of frames of the medical video to a trained feature generator;

obtaining from the trained feature generator a plurality of feature vectors corresponding to the plurality of frames;

providing the plurality of feature vectors to a second trained classifier;

obtaining from the second trained classifier a valve label indicative of a valve depicted in each of the plurality of frames.

24. The method of claim 22 , wherein selecting the frame comprises:

providing a plurality of frames of the medical video to a trained feature generator;

obtaining from the trained feature generator a plurality of feature vectors corresponding to the plurality of frames;

providing the plurality of feature vectors to a second trained classifier;

obtaining from the second trained classifier a mode label indicative of a mode of each of the plurality of frames.