Hardware-implemented argmax layer
A hardware acceleration module may generate a channel-wise argmax map using a predefined set of hardware-implemented operations. In some examples, a hardware acceleration module may receive a set of feature maps for different image channels. The hardware acceleration module may execute a sequence of hardware operations, including a portion(s) of hardware for executing a convolution, rectified linear unit (ReLU) activation, and/or layer concatenation, to determine a maximum channel feature value and/or argument maxima (argmax) value for a set of associated locations within the feature maps. An argmax map may be generated based at least in part on the argument maximum for a set of associated locations.
1. A system comprising:
a central processing unit (CPU), comprising:
one or more processors; and
a memory storing processor-executable instructions that, when executed by the one or more processors, cause the system to perform software operations; and
a hardware acceleration module, comprising a plurality of integrated circuits configured to perform one or more hardware-implemented operations, wherein the hardware acceleration module is configured to:
receive a plurality of feature maps based on image data, each feature map having a first location storing a first feature value based on one or more regions of the image data, wherein each feature map stores one or more feature values for a different channel of the image data, and wherein a first set of associated feature values includes the first feature value stored at the first location within each of the plurality of feature maps;
determine an argument maxima value for the first set of associated feature values, wherein the argument maxima value is determined by executing a sequence of the hardware-implemented operations on the first set of associated feature values;
generate an argument maxima map storing the argument maxima value determined for the first set of associated feature values; and
transmit the argument maxima map to the CPU for processing via the software operations.
2. The system of claim 1 , wherein the CPU and hardware acceleration module comprise different sets of hardware.
3. The system of claim 1 , wherein the hardware-implemented operations implemented by the hardware acceleration module include at least a convolution operation, a rectified linear unit (ReLU) activation operation, and a layer concatenation operation, and wherein the hardware acceleration module.
4. The system of claim 3 , wherein the hardware acceleration module is configured to operate with fixed-point number representations.
5. A hardware acceleration module, comprising:
a plurality of integrated circuits configured to perform one or more hardware-implemented operations, wherein the hardware acceleration module is configured to:
receive
a plurality of feature maps based on image data, each feature map having a first location storing a first feature value based on one or more regions of the image data, wherein each feature map stores one or more feature values for a different channel of the image data, and wherein a first set of associated feature values includes the first feature value stored at the first location within each of the plurality of feature maps;
determine an argument maxima value for the first set of associated feature values, wherein the argument maxima value is determined by executing a sequence of the hardware-implemented operations on the first set of associated feature values; and
generate an argument maxima map storing the argument maxima value determined for the first set of associated feature values.
6. The hardware acceleration module of claim 5 , wherein the hardware-implemented operations implemented by the hardware acceleration module include at least a convolution operation, a rectified linear unit (ReLU) activation operation, and a layer concatenation operation.
7. The hardware acceleration module of claim 6 , wherein the hardware acceleration module is configured to operate with fixed-point number representations.
8. The hardware acceleration module of claim 6 , wherein the hardware acceleration module is further configured to transmit the argument maxima map to a central processing unit (CPU) comprising a separate set of hardware from the hardware acceleration module.
9. The hardware acceleration module of claim 5 , wherein determining the argument maxima value for a first set of associated feature values stored at the first location within each of the plurality of feature maps comprises:
using a first sequence of the hardware-implemented operations implemented by the hardware acceleration module, to determine a maximum of the first set of associated feature values; and
using a second sequence of the hardware-implemented operations implemented by the hardware acceleration module, to determine a first argument maxima value corresponding to the determined maximum of the first set of associated feature values.
10. The hardware acceleration module of claim 9 , wherein the first sequence of hardware-implemented operations used to determine the maximum of the first set of associated feature values comprises:
performing a first hardware-implemented convolution operation, with fixed weights, on two feature values within the first set of associated feature values, to generate a first set of outputs;
applying a hardware-implemented ReLU activation operation to the first set of outputs, to generate a second set of outputs;
performing a hardware-implemented concatenation operation on the second set of outputs and the two feature values, to generate a third set of outputs; and
performing a second hardware-implemented convolution operation to sum the third set of outputs and divide the sum by two.
11. The hardware acceleration module of claim 10 , wherein determining the maximum of the first set of associated feature values comprises performing the first sequence of hardware-implemented operations a number of times depending on the number of feature values within the first set of associated feature values.
12. The hardware acceleration module of claim 11 , wherein the second sequence of operations used to determine the first argument maxima value corresponding to the determined maximum of the first set of associated feature values comprises:
performing a hardware-implemented concatenation operation on the first set of feature values and the determined maximum of the first set of feature values, to generate a fourth set of outputs;
performing a third hardware-implemented convolution operation to subtract the determined maximum of the first set of associated feature values from, and add a positive constant value, each of first set of associated feature values, to generate a fifth set of outputs;
applying a hardware-implemented ReLU activation operation to the fifth set of outputs, to generate a sixth set of outputs; and
performing at least a fourth hardware-implemented convolution operation on the sixth set of outputs, in which each particular output of the sixth set of outputs is multiplied by two to the power of an index value associated with the particular output, to generate a seventh set of outputs, and in which the seventh set of outputs is summed to calculate the first argument maxima value.
13. A method comprising:
receiving, at a hardware acceleration module,
a plurality of feature maps based on image data, each feature map having a first location storing a first feature value based on one or more regions of the image data, wherein each feature map stores one or more feature values for a different channel of the image data, and wherein a first set of associated feature values includes the first feature value stored at the first location within each of the plurality of feature maps;
determining, by the hardware acceleration module, an argument maxima value for the first set of associated feature values, wherein the argument maxima value is determined by executing a sequence of hardware-implemented operations on the first set of associated feature values; and
generating, by the hardware acceleration module, an argument maxima map storing the argument maxima value determined for the first set of associated feature values.
14. The method of claim 13 , wherein the hardware-implemented operations implemented by the hardware acceleration module include at least a convolution operation, a rectified linear unit (ReLU) activation operation, and a layer concatenation operation.
15. The method of claim 14 , wherein the hardware acceleration module is configured to operate with fixed-point number representations.
16. The method of claim 14 , further comprising transmitting the argument maxima map to a central processing unit (CPU) comprising a separate set of hardware from the hardware acceleration module.
17. The method of claim 13 , wherein determining the argument maxima value for a first set of associated feature values stored at the first location within each of the plurality of feature maps comprises:
using a first sequence of the hardware-implemented operations implemented by the hardware acceleration module, to determine a maximum of the first set of associated feature values; and
using a second sequence of the hardware-implemented operations implemented by the hardware acceleration module, to determine a first argument maxima value corresponding to the determined maximum of the first set of associated feature values.
18. The method of claim 17 , wherein the first sequence of hardware-implemented operations used to determine the maximum of the first set of associated feature values comprises:
performing a first hardware-implemented convolution operation, with fixed weights, on two feature values within the first set of associated feature values, to generate a first set of outputs;
applying a hardware-implemented ReLU activation operation to the first set of outputs, to generate a second set of outputs;
performing a hardware-implemented concatenation operation on the second set of outputs and the two feature values, to generate a third set of outputs; and
performing a second hardware-implemented convolution operation to sum the third set of outputs and divide the sum by two.
19. The method of claim 18 , wherein determining the maximum of the first set of associated feature values comprises performing the first sequence of hardware-implemented operations a number of times depending on the number of feature values within the first set of associated feature values.
20. The method of claim 19 , wherein the second sequence of operations used to determine the first argument maxima value corresponding to the determined maximum of the first set of associated feature values comprises:
performing a hardware-implemented concatenation operation on the first set of feature values and the determined maximum of the first set of feature values, to generate a fourth set of outputs;
performing a third hardware-implemented convolution operation to subtract the determined maximum of the first set of associated feature values from, and add a positive constant value, each of first set of associated feature values, to generate a fifth set of outputs;
applying a hardware-implemented ReLU activation operation to the fifth set of outputs, to generate a sixth set of outputs; and
performing at least a fourth hardware-implemented convolution operation on the sixth set of outputs, in which each particular output of the sixth set of outputs is multiplied by two to the power of an index value associated with the particular output, to generate a seventh set of outputs, and in which the seventh set of outputs is summed to calculate the first argument maxima value.