Disease detection with masked attention
A candidate generator generates a set of candidate three-dimensional image patches from an input volume. A candidate classifier classifies the set of candidate three-dimensional image patches as containing or not containing disease. Classifying the set of candidate three-dimensional image patches comprises generating an attention mask for each given candidate three-dimensional image patch within the set of candidate three-dimensional image patches to form a set of attention masks, applying the set of attention masks to the set of candidate three-dimensional image patches to form a set of masked image patches, and classifying the set of masked image patches as containing or not containing the disease. The candidate classifier applies soft attention and hard attention to the three-dimensional image patches such that distinctive image regions are highlighted proportionally to their contribution to classification while completely removing image regions that may cause confusion.
1. A method, in a data processing system, for computer aided disease detection with masked attention, the method comprising:
generating, by a candidate generator, a set of candidate three-dimensional image patches from an input volume;
classifying, by a candidate classifier, the set of candidate three-dimensional image patches as containing or not containing disease, wherein classifying the set of candidate three-dimensional image patches comprises:
generating an attention mask for each given candidate three-dimensional image patch within the set of candidate three-dimensional image patches to form a set of attention masks;
applying the set of attention masks to the set of candidate three-dimensional image patches to form a set of masked image patches; and
classifying the set of masked image patches as containing or not containing the disease,
wherein the candidate classifier applies soft attention and hard attention to the three-dimensional image patches such that distinctive image regions are highlighted proportionally to their contribution to classification while completely removing image regions that may cause confusion, wherein generating the set of candidate three-dimensional image patches comprises decomposing the input volume into a set of image slabs and applying a first neural network to the set of image slabs to segment the set of image slabs into the set of candidate three-dimensional image patches.
2. The method of claim 1 , wherein the first neural network is a two-dimensional UNet neural network.
3. The method of claim 1 , wherein generating the attention mask for each given candidate three-dimensional image patch comprises applying a second neural network and a rectified linear unit (ReLU) to the given candidate three-dimensional image patch.
4. The method of claim 3 , wherein the rectified linear unit applies an ReLU function A(x) as follows:
A
(
x
)
=
{
M
(
x
)
if
M
(
x
)
≥
0
0
otherwise
,
where M(x) is a transform map M∈ L×L×L , L×L×L is the size of the input image patch.
5. The method of claim 1 , wherein a applying the set of attention masks to the set of candidate three-dimensional image patches comprises applying element-wise multiplication.
6. The method of claim 3 , wherein classifying the set of masked image patches comprises applying a third neural network to the set of masked image patches.
7. The method of claim 6 , wherein the third neural network is a three-dimensional squeezeNet.
8. The method of claim 6 , wherein the second neural network and the third neural network are trained jointly.
9. The method of claim 1 , further comprising classifying the input volume as containing the disease if and only if at least one of the set of masked image patches is classified as containing the disease.
10. A computer program product comprising a non-transitory computer readable medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to:
generate, by a candidate generator, a set of candidate three-dimensional image patches from an input volume;
classify, by a candidate classifier, the set of candidate three-dimensional image patches as containing or not containing disease, wherein classifying the set of candidate three-dimensional image patches comprises:
generating an attention mask for each given candidate three-dimensional image patch within the set of candidate three-dimensional image patches to form a set of attention masks;
applying the set of attention masks to the set of candidate three-dimensional image patches to form a set of masked image patches; and
classifying the set of masked image patches as containing or not containing the disease,
wherein the candidate classifier applies soft attention and hard attention to the three-dimensional image patches such that distinctive image regions are highlighted proportionally to their contribution to classification while completely removing image regions that may cause confusion, wherein generating the set of candidate three-dimensional image patches comprises decomposing the input volume into a set of image slabs and applying a first neural network to the set of image slabs to segment the set of image slabs into the set of candidate three-dimensional image patches.
11. The computer program product of claim 10 , wherein generating the attention mask for each given candidate three-dimensional image patch comprises applying a second neural network and a rectified linear unit (ReLU) to the given candidate three-dimensional image patch.
12. The computer program product of claim 11 , wherein the rectified linear unit applies an ReLU function A(x) as follows:
A
(
x
)
=
{
M
(
x
)
if
M
(
x
)
≥
0
0
otherwise
,
where M(x) is a transform map M∈ L×L×L , L×L×L is the size of the input image patch.
13. The computer program product of claim 10 , wherein a applying the set of attention masks to the set of candidate three-dimensional image patches comprises applying element-wise multiplication.
14. The computer program product of claim 13 , wherein classifying the set of masked image patches comprises applying a third neural network to the set of masked image patches.
15. The computer program product of claim 14 , wherein the third neural network is a three-dimensional squeezeNet.
16. The computer program product of claim 14 , wherein the second neural network and the third neural network are trained jointly.
17. The computer program product of claim 10 , further comprising classifying the input volume as containing the disease if and only if at least one of the set of masked image patches is classified as containing the disease.
18. An apparatus comprising:
a processor; and
a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to:
generate, by a candidate generator, a set of candidate three-dimensional image patches from an input volume;
classify, by a candidate classifier, the set of candidate three-dimensional image patches as containing or not containing disease, wherein classifying the set of candidate three-dimensional image patches comprises:
generating an attention mask for each given candidate three-dimensional image patch within the set of candidate three-dimensional image patches to form a set of attention masks;
applying the set of attention masks to the set of candidate three-dimensional image patches to form a set of masked image patches; and
classifying the set of masked image patches as containing or not containing the disease,
wherein the candidate classifier applies soft attention and hard attention to the three-dimensional image patches such that distinctive image regions are highlighted proportionally to their contribution to classification while completely removing image regions that may cause confusion, wherein generating the set of candidate three-dimensional image patches comprises decomposing the input volume into a set of image slabs and applying a first neural network to the set of image slabs to segment the set of image slabs into the set of candidate three-dimensional image patches.
19. The apparatus of claim 18 , wherein generating the attention mask for each given candidate three-dimensional image patch comprises applying a second neural network and a rectified linear unit (ReLU) to the given candidate three-dimensional image patch.
20. The apparatus of claim 19 , wherein classifying the set of masked image patches comprises applying a third neural network to the set of masked image patches.