Systems and methods for hardware-based pooling
Described herein are systems and methods that utilize a novel hardware-based pooling architecture to process the output of a convolution engine representing an output channel of a convolution layer in a convolutional neural network (CNN). The pooling system converts the output into a set of arrays and aligns them according to a pooling operation to generate a pooling result. In certain embodiments, this is accomplished by using an aligner that aligns, e.g., over a number of arithmetic cycles, an array of data in the output into rows and shifts the rows relative to each other. A pooler applies a pooling operation to a combination of a subset of data from each row to generate the pooling result.
1. A pooling unit comprising one or more sub-circuits, wherein the pooling unit is configured to:
reformat input data into a plurality of rows to generate a pooling array, the input data being a linearized array, which includes a plurality of groups of neighborhood values, and a number of the rows corresponding to a size of a pooling kernel, wherein to reformat the pooling unit is configured to shift, over a number of arithmetic cycles, the rows relative to each other to reformat the input data, wherein the input data is received from a matrix processor and reflects an output channel of a convolutional layer, wherein the pooling array is reformatted in grid format such that one or more elements of an individual group are aligned in a horizontal direction and one or more elements of the individual group are aligned in a vertical direction, wherein the rows are shifted according to a shift of a pooling window corresponding to a stride having a value defined by the number of arithmetic cycles and dictating how often pooling values are output,
wherein each row comprises a portion of the input data that correspond to an individual group of neighborhood values of the plurality of groups of neighborhood values, wherein individual pooling operations are applied to individual groups of neighborhood values; and
apply in subsequent arithmetic cycles, according to the reformatted grid format, the individual pooling operations to the individual groups of neighborhood values to obtain a pooling output associated with the output channel, wherein the pooling unit is configured to output a pooling value every number of arithmetic cycles corresponding to the value of the stride.
2. The pooling unit according to claim 1 , wherein the input data has been generated by the matrix processor.
3. The pooling unit according to claim 2 , wherein, to maintain a stream of the input data, the pooling output is generated at a same rate as a rate at which the matrix processor generates the input data.
4. The pooling unit according to claim 2 , wherein pooling calculations are performed in parallel, and wherein a number of pooling calculations equals a number of output channels in the matrix processor, such that the pooling output corresponds to a width of the matrix processor.
5. The pooling unit according to claim 1 , wherein the pooling unit comprises a multiply-and-shift circuit configured to generate the pooling output based on the individual pooling operations.
6. The pooling unit according to claim 1 , wherein the input data corresponds to a set of feature maps, and wherein the reformatted input data is used to reduce, by a predetermined factor, at least one of a height and a width of the set of feature maps.
7. The pooling unit according to claim 1 , wherein the rows have a same width as the input data.
8. The pooling unit according to claim 1 , wherein the pooling unit is configured to use a state machine to shift the pooling output into an output array.
9. The pooling unit according to claim 1 , wherein the pooling unit is further configured to determine, without modifying a sequence of a pooling operation itself, a number and a location of data points involved in the pooling operation.
10. The pooling unit of claim 1 , wherein the plurality of rows is filled with new input data reflecting a new output channel of the convolutional layer after the pooling output is generated.
11. A method for using a hardware-based pooling system, the method comprising:
receiving from a matrix processor a linearized array of data that represents an output channel of a convolution layer in a convolutional neural network (CNN), wherein the linearized array of data comprises a plurality of groups of neighborhood values, wherein the linearized array of data corresponds to a set of feature maps;
converting the linearized array of data into a set of rows in a grid format having a same width as the linearized array of data and such that one or more elements of an individual group are aligned in a horizontal direction and one or more rows of the individual group are aligned in a vertical direction, wherein the rows are shifted according to a shift of a pooling window corresponding to a stride having a value defined by a number of arithmetic cycles and dictating how often pooling values are output, wherein each row comprises portions of data that correspond to an individual group of neighborhood values of the plurality of groups of neighborhood values to which an individual pooling operation is to be applied, wherein a number of the rows corresponds to a size of a pooling kernel, wherein the set of rows are aligned according to a pooling operation that applies data to at least two rows of the set of rows to generate a pooling result according to the grid format; and
outputting the pooling result into a memory device, wherein a pooling value is output every number of arithmetic cycles corresponding to the value of the stride.
12. The method according to claim 11 , wherein the linearized array of data is received at a hardware-based pooling unit.
13. The method according to claim 11 , wherein linearized arrays of data are received at intervals of the number of arithmetic cycles.
14. The method according to claim 11 , wherein pooling results are generated at each interval.
15. The method according to claim 14 , wherein the pooling results are output at each interval.
16. A method for using a pooling unit, the method comprising:
receiving, from a matrix processor at a hardware-based pooling engine, a set of data arrays that each have a predefined relationship with each other;
applying, according to a stride value, a pooling operation to data in at least two arrays from the set of data arrays to obtain a pooling result, wherein the hardware-based pooling engine is configured not to write a convolution result into memory, wherein the set of data arrays are converted into a plurality of rows, wherein the stride value determines a number of shifts between rows corresponding to a shift in a pooling window by defining a number of arithmetic cycles that comprise an interval in which a pooling value is output, wherein the stride value is defined by the number of arithmetic cycles and dictates how often the pooling result is output;
wherein the set of data arrays include portions of data that correspond to an individual group of neighborhood values of a plurality of groups of neighborhood values to which an individual pooling operation is to be applied,
wherein the set of data arrays are in grid format such that one or more elements of the individual group of neighborhood values are aligned in a horizontal direction and one or more elements of the individual group of neighborhood values are aligned in a vertical direction, and
wherein a number of the rows corresponds to a size of the pooling window; and
outputting, according to the grid format, the pooling result as a row of data points which are associated with a layer of a convolutional neural network (CNN), wherein a pooling value is output every number of arithmetic cycles corresponding to the stride value.
17. The method according to claim 16 , wherein obtaining the pooling result utilizes a one-to-one relationship between an output channel and an input channel.
18. The method according to claim 16 , wherein the pooling result comprises one of an average pooling result and a max pooling result.