Implementing fundamental computational primitives using a matrix multiplication accelerator (MMA)
A method for performing a fundamental computational primitive in a device is provided, where the device includes a processor and a matrix multiplication accelerator (MMA). The method includes configuring a streaming engine in the device to stream data for the fundamental computational primitive from memory, configuring the MMA to format the data, and executing the fundamental computational primitive by the device.
1. A matrix multiplication accelerator comprising:
a first formatting component having a first input and a first output;
a second formatting component having a second input and second output;
a third formatting component having a third input and a third output;
a row offset component having an offset input coupled to the second output, and an offset output;
a first memory having a first matrix input coupled to the first output, and a first matrix output;
a second memory having a second matrix input coupled to the offset output, and a second matrix output;
a nonlinearity component having a nonlinearity input, and a nonlinearity output coupled to the third input;
a third memory having a third matrix input, and a third matrix output coupled to the nonlinearity input; and
a matrix multiplication component coupled to the first matrix output, the second matrix output, and the third matrix input.
2. The matrix multiplication accelerator of claim 1 , wherein:
the first input, the second input, and the third output are coupled to a processor having a first streaming engine and a second streaming engine.
3. The matrix multiplication accelerator of claim 1 , wherein:
the first formatting component, the second formatting component, the third formatting component, the row offset component, and the nonlinearity component are each include a configuration input.
4. The matrix multiplication accelerator of claim 3 , wherein:
the first formatting component, the second formatting component, the third formatting component, the row offset component, and the nonlinearity component are each configured based on a configuration setting received on the configuration input.
5. The matrix multiplication accelerator of claim 4 , wherein:
the configuration setting is based on a computational primitive.
6. The matrix multiplication accelerator of claim 5 , wherein:
the computational primitive includes one of a two-dimensional (2D) convolution, a matrix multiplication, a matrix point wise multiplication, a matrix addition, a vector matrix multiplication, a vector summation, an affine transformation, a fast Fourier transform, a discrete cosign transform, a correlation, a matrix assignment, a matrix permutation, and a matrix transposition.
7. The matrix multiplication accelerator of claim 1 , wherein:
the second memory includes two 32×32 multiplicand buffers.
8. The matrix multiplication accelerator of claim 1 , wherein:
the third memory includes two 32×32 result buffers.
9. The matrix multiplication accelerator of claim 1 , wherein:
the matrix multiplication accelerator performs a load operation, a store operation, and an execute operation in a single cycle.
10. The matrix multiplication accelerator of claim 9 , wherein:
the load operation includes a first field identifying a location of a first data to load into the first memory.
11. The matrix multiplication accelerator of claim 9 , wherein:
the load operation includes a second field identifying a location of a second data to load into the second memory.
12. The matrix multiplication accelerator of claim 9 , wherein:
the load operation includes a third field identifying whether the load operation is to be performed.
13. The matrix multiplication accelerator of claim 9 , wherein:
the store operation includes a first field identifying a location in the third memory that stores a matrix multiplication output received on the third matrix input from the matrix multiplication component.
14. The matrix multiplication accelerator of claim 9 , wherein:
the store operation includes a second field identifying whether the store operation is to be performed.
15. The matrix multiplication accelerator of claim 9 , wherein:
the execute operation includes a first field identifying an operation type.
16. The matrix multiplication accelerator of claim 1 , wherein:
the nonlinearity output is zero when the nonlinearity input is negative.
17. The matrix multiplication accelerator of claim 1 , wherein:
the nonlinearity output is set to the nonlinearity input when the nonlinearity input is non-negative.
18. The matrix multiplication accelerator of claim 1 , wherein:
the first memory is arranged for storing filter vectors associated with filters of a convolution layer;
the second memory is arranged for storing input feature maps of the convolution layer; and
the third memory is arranged for storing output feature maps resulting from performing a computational primitive using the filter vectors and the input feature maps.
19. A matrix multiplication accelerator, comprising:
a first formatting component having a first input and a first output;
a second formatting component having a second input and second output;
a third formatting component having a third input and a third output;
a row offset component having an offset input coupled to the second output, and an offset output;
a first memory having a first matrix input coupled to the first output, and a first matrix output;
a second memory having a second matrix input coupled to the offset output, and a second matrix output;
a nonlinearity component having a nonlinearity input, and a nonlinearity output coupled to the third input;
a third memory having a third matrix input, and a third matrix output coupled to the nonlinearity input; and
a matrix multiplication component coupled to the first matrix output, the second matrix output, and the third matrix input;
wherein the first formatting component, the second formatting component, the third formatting component, the row offset component, and the nonlinearity component each include a configuration input; and
wherein the first formatting component, the second formatting component, the third formatting component, the row offset component, and the nonlinearity component are each configured based on a configuration setting received on the configuration input corresponding to a computational primitive.
20. The matrix multiplication accelerator of claim 19 , wherein:
the first memory is arranged for storing filter vectors associated with filters of a convolution layer;
the second memory is arranged for storing input feature maps of the convolution layer; and
the third memory is arranged for storing output feature maps resulting from performing the computational primitive using the filter vectors and the input feature maps.