Method and system of building hospital-scale chest X-ray database for entity extraction and weakly-supervised classification and localization of common thorax diseases
A new chest X-ray database, referred to as “ChestX-ray8”, is disclosed herein, which comprises over 100,000 frontal view X-ray images of over 32,000 unique patients with the text-mined eight disease image labels (where each image can have multi-labels), from the associated radiological reports using natural language processing. We demonstrate that these commonly occurring thoracic diseases can be detected and spatially-located via a unified weakly supervised multi-label image classification and disease localization framework, which is validated using our disclosed dataset.
1. A method comprising:
providing a chest x-ray image of a subject; and
by a computing system, analyzing the chest x-ray image of the subject by applying a unified weakly supervised multi-label image classification and disease localization framework to the chest x-ray image of the subject and determining a presence and anatomical locations of one or more thoracic diseases in the subject based on the analysis of the chest x-ray image wherein the unified weakly supervised multi-label image classification and disease localization framework comprises a multi-label DCNN classification model, wherein the DCNN comprises a transition layer, a global pooling layer, a prediction layer, and a multi-label classification loss layer in an end after a last convolutional layer.
2. The method of claim 1 , wherein the anatomical locations of the one or more thoracic diseases are identified with one or more bounding boxes localized relative to the chest x-ray images.
3. The method of claim 2 , further comprising generating an automated radiological report corresponding to the x-ray image of the subject based on the determined presence and anatomical locations of the one or more thoracic diseases in the subject.
4. The method of claim 2 , wherein the determining utilizes a database containing a plurality of chest x-ray images and a plurality of corresponding thoracic diseases and locations associated with the plurality of chest x-ray images, the plurality of corresponding thoracic diseases and locations having been text-mined from radiological reports corresponding to the plurality of chest x-ray images using natural language processing.
5. The method of claim 2 , wherein the one or more thoracic diseases comprise Atelectasis, Cardiomegaly, Effusion, Infiltration, Mass, Nodule, Pneumonia, and Pneumathorax.
6. The method of claim 1 , wherein Deep Convolutional Neural Network (DCNN) architectures are used for the weakly-supervised disease localization.
7. The method of claim 1 , wherein spatial locations of diseases are determined using a combination of deep activations from the transition layer and weights of the prediction layer.
8. The method of claim 1 , wherein the transition layer transforms the activations from previous layers into a uniform dimension of output.
9. A computer readable storage device comprising computer-executable instructions for performing the method of claim 1 .
10. The method of claim 1 , further comprising treating the subject for the determined thoracic diseases.
11. A system comprising:
a computer processor; and
a data storage device that stores a database comprising:
a plurality of chest x-ray images; and
disease labels and locations associated with each chest x-ray image;
wherein the disease labels and locations are mined from radiological reports associated with the chest x-ray images, the disease labels identifying thoracic diseases indicated by the corresponding chest x-ray images, and the locations identifying anatomical areas in the chest x-ray images where the indicated thoracic diseases are located;
wherein the computer processor is operable to receive or generate a chest x-ray image of a subject and to determine presence and anatomical locations of one or more thoracic diseases in the subject based on automated analysis of the chest x-ray image of the subject using a unified weakly supervised multi-label image classification and disease localization framework wherein the unified weakly supervised multi-label image classification and disease localization framework comprises a multi-label DCNN classification model, wherein the DCNN comprises a transition layer, a global pooling layer, a prediction layer and a multi-label classification loss layer.
12. The system of claim 11 , wherein the system is further operable to generate a radiological report corresponding to the x-ray image of the subject based on the determined presence and anatomical locations of the one or more thoracic diseases in the subject.
13. The system of claim 11 , wherein the thoracic diseases comprise Atelectasis, Cardiomegaly, Effusion, Infiltration, Mass, Nodule, Pneumonia, and Pneumathorax.
14. The system of claim 11 , wherein the anatomical locations of the one or more thoracic diseases are identified with one or more bounding boxes localized relative to the chest x-ray images.
15. The system of claim 11 , wherein the system uses Deep Convolutional Neural Network (DCNN) architectures for weakly-supervised object localization.
16. The system of claim 11 , wherein spatial locations of diseases are determined using a combination of deep activations from the transition layer and weights of the prediction layer.
17. The system of claim 11 , further comprising an x-ray imaging device that generates the chest x-ray of the subject.
18. A computer readable storage device comprising the database of claim 11 .