Implementing fundamental computational primitives using a matrix multiplication accelerator (MMA)
A method for performing a fundamental computational primitive in a device is provided, where the device includes a processor and a matrix multiplication accelerator (MMA). The method includes configuring a streaming engine in the device to stream data for the fundamental computational primitive from memory, configuring the MMA to format the data, and executing the fundamental computational primitive by the device.
1. A method comprising:
configuring, by a processor, a streaming engine, based on a computational primitive, to stream data vectors, wherein the data vectors include data elements;
formatting, by the processor, the data vectors based on the computational primitive;
configuring, by the processor, a matrix multiplication accelerator (MMA) based on the computational primitive; and
executing, by the MMA, the computational primitive to produce an MMA output.
2. The method of claim 1 , wherein executing the computational primitive further comprises loading data into a first multiplicand matrix buffer concurrently with execution of matrix multiplication using data in a second multiplicand matrix buffer, the first and second multiplicand matrix buffers comprised in the MMA.
3. The method of claim 1 , wherein configuring the MMA further comprises configuring the MMA to offset load data in a multiplicand matrix buffer comprised in the MMA.
4. The method of claim 1 , wherein the data comprises a row of a matrix, and wherein executing the computational primitive further comprises copying the data to a column in a multiplicand matrix buffer comprised in the MMA.
5. The method of claim 1 , further comprising formatting, by the processor, the MMA output to add zero padding to the MMA output.
6. The method of claim 1 , further comprising formatting, by the processor, the MMA output to perform seam removal on the MMA output.
7. The method of claim 1 , further comprising formatting, by the processor, the MMA output to perform column subsampling on the MMA output according to a specified stride.
8. The method of claim 1 , wherein configuring the streaming engine further comprises configuring the streaming engine to tile data of a plurality of feature maps into a filtering matrix.
9. The method of claim 8 , wherein a size of a tile of data is selected such that no seam is present in the output of the MMA.
10. The method of claim 8 , wherein a size of a tile of data is selected such that a seam is always in a same location in the output of the MMA.
11. The method of claim 1 , wherein the data vectors include interleaved real and imaginary values; and
wherein the MMA is configured to perform matrix multiplication on the data vectors.
12. The method of claim 1 , wherein the data vectors comprise a filter of a length greater than a width of a multiplicand matrix buffer comprised in the MMA, and the filter is decomposed into a plurality of smaller filters, wherein sizes of the smaller filters are chosen as a tradeoff between maximizing the throughput of the MMA and minimizing a total number of smaller filters.
13. A device comprising:
a memory;
a processor coupled to the memory; and
a matrix multiplication accelerator (MMA) coupled to the processor, the MMA comprising a multiplier buffer and a first multiplicand buffer,
wherein the processor is operable to:
configure a streaming engine comprised in the device to stream data vectors based on a computational primitive, wherein the data vectors include data elements;
format the data vectors based on the computational primitive; and
configure the MMA based on the computational primitive; and
wherein the MMA is operable to execute the computational primitive to produce an MMA output.
14. The device of claim 13 , wherein the device is further operable to execute the computational primitive wherein data is loaded into the first multiplicand matrix buffer concurrently with execution of matrix multiplication using data in a second multiplicand matrix buffer comprised in the MMA.
15. The device of claim 13 , wherein the device is further operable to configure the MMA to offset load data in the first multiplicand matrix buffer.
16. The device of claim 13 , wherein the device is further operable to execute the computational primitive, wherein the data comprises a row of a matrix, and the data is copied to a column in the first multiplicand matrix buffer.
17. The device of claim 13 , wherein the device is further operable to configure the processor to format the MMA output to add zero padding to the MMA output.
18. The device of claim 13 , wherein the device is further operable to configure the processor to format the MMA output to perform seam removal on the MMA output.
19. The device of claim 13 , wherein the device is further operable to configure the processor to format the MMA output to perform column sub sampling on the MMA output according to a specified stride.
20. The device of claim 13 , wherein the device is further operable to configure the streaming engine to tile data of a plurality of feature maps into a filtering matrix.
21. The device of claim 20 , wherein a size of a tile of data is selected such that no seam is present in output of the MMA.
22. The device of claim 20 , wherein a size of a tile of data is selected such that a seam is always in a same location in output of the MMA.
23. The device of claim 13 , wherein the data vectors include interleaved real and imaginary values; and
wherein the MMA is configured to perform matrix multiplication on the data vectors.
24. The device of claim 13 , wherein the data vectors comprise a filter of a length greater than a width of the first multiplicand matrix buffer, and the device is further operable to decompose the filter into a plurality of smaller filters, wherein sizes of the smaller filters are chosen as a tradeoff between maximizing the throughput of the MMA and minimizing a total number of the smaller filters.