Ventral-dorsal neural networks: object detection via selective attention
Embodiments described herein relate generally to a methodology of efficient object classification within a visual medium. The methodology utilizes a first neural network to perform an attention based object localization within a visual medium to generate a visual mask. The visual mask is applied to the visual medium to generate a masked visual medium. The masked visual medium may be then fed into a second neural network to detect and classify objects within the visual medium.
1. A computer-implemented method for object detection within a visual medium, comprising:
receiving a visual medium comprising a plurality of objects;
identifying, via a first neural network, one or more relevant visual regions and one or more irrelevant visual regions within the visual medium, comprising:
identifying, via a sensitivity analysis, pixels within the visual medium that are above a predetermined threshold, wherein the pixels above the predetermined threshold define the one or more relevant visual regions;
generating, based at least on the one or more irrelevant visual regions, a visual mask comprising a data structure containing pixel values;
applying the visual mask to modify pixel intensity values of the one or more irrelevant visual regions to generate a masked visual medium;
identifying, via a second neural network, one or more objects of interest within the masked visual medium; and
outputting an identification of the one or more objects of interest.
2. The computer-implemented method of claim 1 , wherein pixel intensity values associated with the one or more relevant visual regions are non-zero.
3. The computer-implemented method of claim 1 , wherein pixel intensity values associated with the one or more irrelevant visual regions are zero.
4. The computer-implemented method of claim 1 , wherein the first neural network is a deep convolutional attention based object detection neural network.
5. The computer-implemented method of claim 1 , wherein the second neural network is a supervised object detection neural network.
6. The computer-implemented method of claim 1 , wherein identifying, via the first neural network, one or more relevant visual regions and one or more irrelevant visual regions within the visual medium further comprises:
extracting convolutional features from the visual medium and aggregating the extracted convolutional features into a Gestalt Total output.
7. The computer-implemented method of claim 1 , wherein the second neural network is utilized on the masked visual medium portion of the visual medium.
8. A non-transitory computer-readable storage medium having stored thereon instructions for causing at least one computer system to detect objects within a visual medium, the instructions comprising:
receiving a visual medium comprising a plurality of objects;
identifying, via a first neural network, one or more relevant visual regions and one or more irrelevant visual regions within the visual medium, comprising:
identifying, via a sensitivity analysis, pixels within the visual medium that are above a predetermined threshold, wherein the pixels above the predetermined threshold define the one or more relevant visual regions;
generating, based at least on the one or more irrelevant visual regions, a visual mask comprising a data structure containing pixel values;
applying the visual mask to modify pixel intensity values of the one or more irrelevant visual regions to generate a masked visual medium;
identifying, via a second neural network, one or more objects of interest within the masked visual medium; and
outputting an identification of the one or more objects of interest.
9. The non-transitory computer-readable storage medium of claim 8 , wherein pixel intensity values associated with the one or more relevant visual regions are non-zero.
10. The non-transitory computer-readable storage medium of claim 8 , wherein pixel intensity values associated with the one or more irrelevant visual regions are zero.
11. The non-transitory computer-readable storage medium of claim 8 , wherein the first neural network is a deep convolutional attention based object detection neural network.
12. The non-transitory computer-readable storage medium of claim 8 , wherein the second neural network is a supervised object detection neural network.
13. The non-transitory computer-readable storage medium of claim 8 , wherein identifying, via the first neural network, one or more relevant visual regions and one or more irrelevant visual regions within the visual medium further comprises:
extracting convolutional features from the visual medium and aggregating the extracted convolutional features into a Gestalt Total output.
14. The non-transitory computer-readable storage medium of claim 8 , wherein the second neural network is only utilized on the masked visual medium portion of the visual medium.
15. A system for detecting objects within a visual medium, comprising:
one or more processors; and
a memory coupled with the one or more processors, the memory configured to store instructions that when executed by the one or more processors cause the one or more processors to:
receive a visual medium comprising a plurality of objects;
identify, via a first neural network, one or more relevant visual regions and one or more irrelevant visual regions within the visual medium, comprising:
identify, via a sensitivity analysis, pixels within the visual medium that are above a predetermined threshold, wherein the pixels above the predetermined threshold define the one or more relevant visual regions;
generate, based at least on the one or more irrelevant visual regions, a visual mask comprising a data structure containing pixel values;
apply the visual mask to modify pixel intensity values of the one or more irrelevant visual regions to generate a masked visual medium;
identify, via a second neural network, one or more objects of interest within the masked visual medium; and
output an identification of the one or more objects of interest.
16. The system of claim 15 , wherein pixel intensity values associated with the one or more relevant visual regions are non-zero.
17. The system of claim 15 , wherein pixel intensity values associated with the one or more irrelevant visual regions are zero.
18. The system of claim 15 , wherein the first neural network is a deep convolutional attention based object detection neural network.
19. The system of claim 15 , wherein the second neural network is a supervised object detection neural network.
20. The system of claim 15 , wherein identify, via the first neural network, one or more relevant visual regions and one or more irrelevant visual regions within the visual medium further comprises:
extract convolutional features from the visual medium and aggregating the extracted convolutional features into a Gestalt Total output.