Explainable visual attention for deep learning
Systems or techniques that facilitate explainable visual attention for deep learning are provided. In various embodiments, a system can access a medical image generated by a medical imaging scanner. In various aspects, the system can perform, via execution of a deep learning neural network, an inferencing task on the medical image. In various instances, the deep learning neural network can receive as input the medical image and can produce as output both an inferencing task result and an attention map indicating on which pixels or voxels of the medical image the deep learning neural network focused in generating the inferencing task result.
1 . A system, comprising:
a processor that executes computer-executable instructions stored in a non-transitory computer-readable memory, wherein execution of the computer-executable instructions causes the processor to:
access a deep learning neural network that comprises a first processing channel and a second processing channel;
access an annotated training dataset that comprises a training medical image, a ground-truth inferencing task result corresponding to the training medical image, and a ground-truth attention map indicating which pixels or voxels of the training medical image should be focused on to produce the ground-truth inferencing task result;
randomly initialize first trainable internal parameters of the first processing channel and second trainable internal parameters of the second processing channel;
execute the deep learning neural network on the training medical image, thereby causing the first processing channel to produce a first output and causing the second processing channel to produce a second output;
compute a first loss between the ground-truth inferencing task result and the first output;
weight the first loss via point-wise multiplication with the ground-truth attention map;
update the first trainable internal parameters of the first processing channel, via backpropagation driven by the weighted first loss;
compute a second loss between the ground-truth attention map and the second output; and
update the second trainable internal parameters of the second processing channel, via backpropagation driven by the second loss.
2 . The system of claim 1 , wherein execution of the computer-executable instructions further causes the processor to:
access a medical image generated by a medical imaging scanner;
execute, after training, the deep learning neural network on the medical image, wherein the deep learning neural network receives the medical image, wherein the first processing channel produces an inferencing task result, and wherein the second processing channel produces an attention map indicating on which pixels or voxels of the medical image the deep learning neural network focused in generating the inferencing task result; and
visually render the inferencing task result and the attention map on an electronic display.
3 . The system of claim 1 wherein the ground-truth attention map is generated based on computing a difference array between the training medical image and the ground-truth inferencing task result.
4 . The system of claim 1 , wherein the ground-truth attention map is generated via execution of an edge detector on the training medical image.
5 . The system of claim 1 , wherein the ground-truth attention map is generated based on computing a difference array between the ground-truth inferencing task result and an output array produced by the first processing channel during a previous training epoch.
6 . A computer-implemented method, comprising:
accessing, by a device operatively coupled to a processor, a deep learning neural network that comprises a first processing channel and a second processing channel;
accessing, by the device, an annotated training dataset that comprises a training medical image, a ground-truth inferencing task result corresponding to the training medical image, and a ground-truth attention map indicating which pixels or voxels of the training medical image should be focused on to produce the ground-truth inferencing task result;
randomly initializing, by the device, first trainable internal parameters of the first processing channel and second trainable internal parameters of the second processing channel;
executing, by the device, the deep learning neural network on the training medical image, thereby causing the first processing channel to produce a first output and causing the second processing channel to produce a second output;
computing, by the device, a first loss between the ground-truth inferencing task result and the first output;
weighting, by the device, the first loss via point-wise multiplication with the ground-truth attention map;
updating, by the device, the first trainable internal parameters of the first processing channel, via backpropagation driven by the weighted first loss;
computing, by the device, a second loss between the ground-truth attention map and the second output; and
updating, by the device, the second trainable internal parameters of the second processing channel, via backpropagation driven by the second loss.
7 . The computer-implemented method of claim 6 , further comprising:
accessing, by the device, a medical image generated by a medical imaging scanner;
executing, by the device and after training, the deep learning neural network on the medical image, wherein the deep learning neural network receives the medical image, wherein the first processing channel produces an inferencing task result, and wherein the second processing channel produces an attention map indicating on which pixels or voxels of the medical image the deep learning neural network focused in generating the inferencing task result; and
visually rendering, by the device, the inferencing task result and the attention map on an electronic display.
8 . The computer-implemented method of claim 6 , wherein the ground-truth attention map is generated based on computing a difference array between the training medical image and the ground-truth inferencing task result.
9 . The computer-implemented method of claim 6 , wherein the ground-truth attention map is generated via execution of an edge detector on the training medical image.
10 . The computer-implemented method of claim 6 , wherein the ground-truth attention map is generated based on computing a difference array between the ground-truth inferencing task result and an output array produced by the first processing channel during a previous training epoch.
11 . A computer program product for facilitating explainable visual attention for deep learning, the computer program product comprising a non-transitory computer-readable memory having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:
access a deep learning neural network that comprises a first processing channel and a second processing channel;
access an annotated training dataset that comprises a training medical image, a ground-truth inferencing task result corresponding to the training medical image, and a ground-truth attention map indicating which pixels or voxels of the training medical image should be focused on to produce the ground-truth inferencing task result;
randomly initialize first trainable internal parameters of the first processing channel and second trainable internal parameters of the second processing channel;
execute the deep learning neural network on the training medical image, thereby causing the first processing channel to produce a first output and causing the second processing channel to produce a second output;
compute a first loss between the ground-truth inferencing task result and the first output;
weight the first loss via point-wise multiplication with the ground-truth attention map;
update the first trainable internal parameters of the first processing channel, via backpropagation driven by the weighted first loss;
compute a second loss between the ground-truth attention map and the second output; and
update the second trainable internal parameters of the second processing channel, via backpropagation driven by the second loss.
12 . The computer program product of claim 11 , wherein the ground-truth attention map is generated based on computing a difference array between the training medical image and the ground-truth inferencing task result.
13 . The computer program product of claim 11 , wherein the ground-truth attention map is generated via execution of an edge detector on the training medical image.
14 . The computer program product of claim 11 , wherein the ground-truth attention map is generated based on computing a difference array between the ground-truth inferencing task result and an output array produced by the first processing channel during a previous training epoch.