PERFORMING POOLING OPERATIONS
A coprocessor of a processing device used to perform pooling operations. The disclosed coprocessor, among other things, receives an activation tensor of a neural network model. The coprocessor applies a pooling window to the activation tensor. The coprocessor reads, from a memory device, a plurality of memory locations. For each memory location, the coprocessor stores a value from a respective memory location associated with a channel of the activation tensor to a corresponding buffer of a set of buffers. Responsive to determining a number of values in each buffer of the set of buffers matches a number of values within the pooling window applied to the activation tensor performing, for each buffer using its values, a pooling operation.
1 . A method comprising:
receiving, by a coprocessor of a processing device, a command to performing pooling operations on an activation tensor of a neural network model, wherein the activation tensor includes a set of channels;
applying a pooling window to the activation tensor;
obtaining, from a plurality of memory locations of a memory device, one or more values present in the pooling window applied to the activation tensor;
for each memory location of the plurality of memory locations, storing a value associated with a channel of set of channels into a corresponding buffer of a set of buffers, wherein each buffer is associated with a channel of the activation tensor;
responsive to determining that each buffer of the set of buffers contains the values within the pooling window when applied to a corresponding channel of the activation tensor, performing, for each buffer, a pooling operation using the values of a respective buffer.
2 . The method of claim 1 , further comprising:
combining an output of each pooling operation; and
storing the combined output in a memory location of the plurality of memory locations.
3 . The method of claim 1 , wherein applying the pooling window to the activation tensor comprises:
moving the pooling window having a predefined size across the activation tensor by a predefined stride in at least one dimension of the activation tensor between successive performance of the pooling operation for the set of buffers.
4 . The method of claim 1 , wherein the activation tensor is stored in the plurality of memory locations using a channel-last format.
5 . The method of claim 1 , wherein an output of the pooling operations a maximum value from the values of the respective buffer.
6 . The method of claim 1 , wherein an output of the pooling operations an average of the values of the respective buffer.
7 . The method of claim 1 , wherein the neural network model is a deep neural network.
8 . A coprocessor coupled to a processing device and a memory device, wherein the coprocessor is to perform operations comprising:
receiving, from the processing device, a command to performing pooling operations on an activation tensor of a neural network model, wherein the activation tensor includes a set of channels;
applying a pooling window to the activation tensor;
obtaining, from a plurality of memory locations of a memory device, one or more values present in the pooling window applied to the activation tensor;
for each memory location of the plurality of memory locations, storing a value associated with a channel of set of channels into a corresponding buffer of a set of buffers, wherein each buffer is associated with a channel of the activation tensor;
responsive to determining that each buffer of the set of buffers contains the values within the pooling window when applied to a corresponding channel of the activation tensor, performing, for each buffer, a pooling operation using the values of a respective buffer.
9 . The coprocessor of claim 8 , wherein the coprocessor is to perform operations further comprising:
combining an output of each pooling operation; and
storing the combined output in a memory location of the plurality of memory locations.
10 . The coprocessor of claim 8 , wherein applying the pooling window to the activation tensor comprises:
moving the pooling window having a predefined size across the activation tensor by a predefined stride in at least one dimension of the activation tensor between successive performance of the pooling operation for the set of buffers.
11 . The coprocessor of claim 8 , wherein the activation tensor is stored in the plurality of memory locations using a channel-last format.
12 . The coprocessor of claim 8 , wherein an output of the pooling operations a maximum value from the values of the respective buffer.
13 . The coprocessor rof claim 8 , wherein an output of the pooling operations an average of the values of the respective buffer.
14 . The coprocessor of claim 8 , wherein the neural network model is a deep neural network.
15 . A system comprising:
a memory device;
a processing device; and
a coprocessor, wherein the coprocessor and the processing device is coupled to the memory device, and wherein the coprocessor is to perform operations comprising:
receiving, from a processing device coupled to the coprocessor, a command to performing pooling operations on an activation tensor of a neural network model, wherein the activation tensor includes a set of channels;
applying a pooling window to the activation tensor;
obtaining, from a plurality of memory locations of a memory device, one or more values present in the pooling window applied to the activation tensor;
for each memory location of the plurality of memory locations, storing a value associated with a channel of set of channels into a corresponding buffer of a set of buffers, wherein each buffer is associated with a channel of the activation tensor;
responsive to determining that each buffer of the set of buffers contains the values within the pooling window when applied to a corresponding channel of the activation tensor, performing, for each buffer, a pooling operation using the values of a respective buffer.
16 . The system of claim 15 , wherein the coprocessor is to perform operations further comprising:
combining an output of each pooling operation; and
storing the combined output in a memory location of the plurality of memory locations.
17 . The system of claim 15 , wherein applying the pooling window to the activation tensor comprises:
moving the pooling window having a predefined size across the activation tensor by a predefined stride in at least one dimension of the activation tensor between successive performance of the pooling operation for the set of buffers.
18 . The system of claim 15 , wherein the activation tensor is stored in the plurality of memory locations using a channel-last format.
19 . The system of claim 15 , wherein an output of the pooling operation is one of: a maximum value from the values of the respective buffer or an average of the values of the respective buffer.
20 . The system of claim 15 , wherein the neural network model is a deep neural network.