Automatic detection and semantic description of lesions using a convolutional neural network
An example system includes a processor to train a convolutional neural network (CNN) to detect features, and train fully connected layers of the CNN to map detected features to semantic descriptors, based on a data set including one or more lesions. The processor is to also receive a medical image to be analyzed for lesions. The processor is to further extract feature maps from the medical image using the trained CNN. The processor is also to detect a region of interest via the trained CNN and generate a bounding box around the detected region of interest. The processor is to reduce a dimension of the region of interest based on the feature maps. The processor is to generate a semantic description of the region of interest via the trained fully connected layers.
1. A system, comprising a processor to:
train a convolutional neural network (CNN) to detect features, and train fully connected layers of the CNN to map detected features to semantic descriptors, based on a data set comprising one or more lesions;
receive a medical image to be analyzed for lesions;
extract feature maps comprising detected features from the medical image using the trained CNN;
detect a region of interest via the trained CNN and generate a bounding box around the detected region of interest;
reduce a dimension of the region of interest based on the feature maps; and
generate a semantic description of the region of interest via the trained fully connected layers.
2. The system of claim 1 , wherein the processor is to calculate a probability that the region of interest comprises a tumor.
3. The system of claim 1 , wherein the processor is to filter out less probable regions of interest based on calculated probabilities that a plurality of regions of interest comprise tumors, wherein the region of interest comprises a higher probability than other regions of interests in the plurality of regions of interest.
4. The system of claim 1 , wherein the feature maps comprise one or more detected features.
5. The system of claim 1 , wherein the processor is to rank the bounding box with other bounding boxes corresponding to other detected regions of interest, wherein the ranking is based on a calculated probability that each detected region of interest comprises a tumor.
6. The system of claim 1 , wherein the semantic description comprises one or more semantic descriptors comprising a shape, a boundary type, a density, or any combination thereof.
7. The system of claim 1 , wherein the processor is to generate and display a diagnostic image comprising the medical image, the bounding box, and the semantic description.
8. A computer-implemented method, comprising:
training, via a processor, a convolutional neural network (CNN) to detect features, and training fully connected layers of the CNN to map the detected features to semantic descriptors, based on a data set comprising one or more lesions;
receiving, via the processor, a medical image to be analyzed for lesions;
extracting, via the processor, feature maps comprising detected features from medical image using the trained CNN;
detecting, via the processor, a region of interest via the trained CNN and generate a bounding box around the detected region of interest;
reducing, via the processor, a dimension of the region of interest based on the feature maps; and
generating, via the processor, a semantic description of the region of interest via the trained fully connected layers.
9. The computer-implemented method of claim 8 , wherein training the CNN and training the fully connected layers is performed concurrently using a batch of positive and negative region of interest candidates.
10. The computer-implemented method of claim 8 , comprising generating, via the processor, a diagnostic image based on the medical image, the bounding box, and the semantic description.
11. The computer-implemented method of claim 8 , comprising generating, via the processor, a score for the bounding box.
12. The computer-implemented method of claim 8 , wherein extracting the feature maps comprises extracting a feature map for each dimension of the medical image.
13. The computer-implemented method of claim 8 , wherein reducing the dimension of the region of interest comprises filtering out less probable regions of interest based on calculated probabilities that a plurality of regions of interest comprise lesions.
14. The computer-implemented method of claim 8 , wherein generating the semantic description of the region of interest comprises populating a template description with one or more semantic descriptor values.
15. A computer program product for detecting and describing lesions automatically, the computer program product comprising a computer-readable storage medium having program code embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program code executable by a processor to cause the processor to:
train a convolutional neural network (CNN) to detect features, and train fully connected layers of the CNN, to map detected features to semantic descriptors based on a data set comprising one or more lesions;
receive a medical image to be analyzed for lesions;
extract feature maps comprising detected features from medical image using the trained CNN;
detect a region of interest via the trained CNN and generate a bounding box around the detected region of interest;
reduce a dimension of the region of interest based on the feature maps; and
generate a semantic description of the region of interest via the trained fully connected layers; and
generate and display a diagnostic image comprising the medical image, the bounding box, and the semantic description.
16. The computer program product of claim 15 , comprising program code executable by the processor to calculate a probability that the region of interest comprises a tumor.
17. The computer program product of claim 15 , comprising program code executable by the processor to:
detect a plurality of regions of interest;
calculate a probability for each region of interest in the plurality of regions of interest;
rank the plurality of regions of interest based on calculated probability; and
generate and display the diagnostic image with a predetermined number of regions of interest with a ranking above a threshold value.
18. The computer program product of claim 15 , comprising program code executable by the processor to train the CNN and train the fully connected layers concurrently using a batch of positive and negative region of interest candidates.
19. The computer program product of claim 15 , comprising program code executable by the processor to extract a feature map for each dimension of the medical image.
20. The computer program product of claim 15 , comprising program code executable by the processor to populate a template description with one or more semantic descriptor values.