Negative sample enhanced object detection machine
A method and system of detecting abnormalities captured in a medical image are provided. A medical image is received having one or more lesions and one or more interferences. One or more convolution layers are applied to the medical image to create one or more feature maps, respectively. A region proposal network (RPN) is applied to each of the one or more feature maps to create a bounding box around each lesion and interference. For each bounding box, an object in the bounding box is classified as a lesion, an interference, or a background. Each object that is incorrectly classified as a lesion but determined to be an interference, is stored as a hard-negative example to be part of a training set for a next application of the RPN.
1. A computing device comprising:
a processor;
a network interface coupled to the processor to enable communication over a network;
a storage device for content and programming coupled to the processor;
a classification engine stored in the storage device, wherein an execution of the classification engine by the processor configures the user device to perform acts comprising:
receiving a medical image having one or more lesions and one or more interferences;
applying one or more convolution layers to the medical image to create one or more feature maps, respectively;
applying a region proposal network (RPN) to each of the one or more feature maps to create a bounding box around each lesion and interference;
for each bounding box, classifying an object in the bounding box as a lesion, an interference, or a background; and
storing each object that is incorrectly classified as a lesion but determined to be an interference, as a hard-negative example to be part of a training set for a subsequent application of the RPN.
2. The computing device of claim 1 , wherein the classification engine is based on Faster R-CNN.
3. The computing device of claim 1 , wherein execution of the classification engine further configures the computing device to perform an act comprising, applying a region of interest (ROI) pooling layer to the one or more feature maps to combine outputs of the region proposal network into a single layer.
4. The computing device of claim 3 , wherein the ROI pooling layer uses maximum pooling.
5. The computing device of claim 1 , wherein the computing device is trained with one or more examples of lesions, one or more examples of background, and one or more examples of an interference.
6. The computing device of claim 1 , wherein the one or more examples of the background are based on a random selection of regions that are determined not to include a lesion or an interference.
7. The computing device of claim 1 , wherein execution of the classification engine further configures the computing device to perform acts comprising, for each bounding box, applying a bounding box regression loss to refine a center point and dimensions of the bounding box to better capture an object therein.
8. The computing device of claim 1 , wherein execution of the classification engine further configures the computing device to perform acts comprising, storing each object that is incorrectly classified as a lesion but determined to be a background, as a hard-negative example to be part of a training set for a subsequent application of the RPN.
9. The computing device of claim 1 , wherein the determination that an object that is an interference is incorrectly classified as an object is based on an intersection over union (IoU) overlap between the object and a ground-truth box being above a predetermined threshold.
10. A non-transitory computer readable storage medium tangibly embodying a computer readable program code having computer readable instructions that, when executed, causes a computing device to carry out a method of detecting abnormalities captured in a medical image, the method comprising:
receiving a medical image having one or more lesions and one or more interferences;
applying one or more convolution layers to the medical image to create one or more feature maps, respectively;
applying a region proposal network (RPN) to each of the one or more feature maps to create a bounding box around each lesion and interference;
for each bounding box, classifying an object in the bounding box as a lesion, an interference, or a background; and
storing each object that is incorrectly classified as a lesion but determined to be an interference, as a hard-negative example to be part of a training set for a subsequent application of the RPN.
11. The non-transitory computer readable storage medium of claim 10 , wherein the classification is based on Faster R-CNN.
12. The non-transitory computer readable storage medium of claim 10 , further comprising, applying a region of interest (ROI) pooling layer to the one or more feature maps to combine outputs of the region proposal network into a single layer.
13. The non-transitory computer readable storage medium of claim 10 , wherein the computer readable program code is trained with one or more examples of lesions, one or more examples of background, and one or more examples of an interference.
14. The non-transitory computer readable storage medium of claim 10 , wherein the one or more examples of the background are based on a random selection of regions that are determined not to include a lesion or an interference.
15. The non-transitory computer readable storage medium of claim 10 , further comprising, for each bounding box, applying a bounding box regression loss to refine a center point and dimensions of the bounding box to better capture an object therein.
16. A computer implemented method, comprising:
receiving a medical image having one or more lesions and one or more interferences;
applying one or more convolution layers to the medical image to create one or more feature maps, respectively;
applying a region proposal network (RPN) to each of the one or more feature maps to create a bounding box around each lesion and interference;
for each bounding box, classifying an object in the bounding box as a lesion, an interference, or a background; and
storing each object that is incorrectly classified as a lesion but determined to be an interference, as a hard-negative example to be part of a training set for a subsequent application of the RPN.
17. The computer implemented method of claim 16 , further comprising, applying a region of interest (ROI) pooling layer to the one or more feature maps to combine outputs of the region proposal network into a single layer.
18. The computer implemented method of claim 16 , wherein the computer implemented method is trained with one or more examples of lesions, one or more examples of background, and one or more examples of an interference.
19. The computer implemented method of claim 16 , wherein the one or more examples of the background are based on a random selection of regions that are determined not to include a lesion or an interference.
20. The computer implemented method of claim 16 , further comprising, for each bounding box, applying a bounding box regression loss to refine a center point and dimensions of the bounding box to better capture an object therein.