Hardware accelerated machine learning
A machine learning hardware accelerator architecture and associated techniques are disclosed. The architecture features multiple memory banks of very wide SRAM that may be concurrently accessed by a large number of parallel operational units. Each operational unit supports an instruction set specific to machine learning, including optimizations for performing tensor operations and convolutions. Optimized addressing, an optimized shift reader and variations on a multicast network that permutes and copies data and associates with an operational unit that support those operations are also disclosed.
1 . An apparatus comprising:
a hardware accelerator, comprising:
a plurality of circuitries to execute instructions associated with machine learning operations;
a local memory comprising a plurality of dual-ported static random-access memory (SRAM), banks to store data associated with one or more of the instructions;
circuitry to permute a first plurality of source data elements associated with a first multidimensional array in accordance with a permutation pattern included with a matrix multiplication instruction and route the first plurality of source data elements to the plurality of circuitries;
the plurality of circuitries, coupled to the circuitry to obtain the first plurality of source data elements, to perform a plurality of parallel multiply-accumulate (MAC) operations in accordance with the matrix multiplication instruction, each of the plurality of parallel MAC operations comprising:
multiplying a first source data element of the first plurality of source data elements provided by the circuitry to permute and a second source data element of a second plurality of source data elements associated with a second multidimensional array to generate a product, and
adding the product to an accumulation value to generate a result value, the first source data element and the second source data element each having a first bit width and the accumulation value having a second bit width at least twice the first bit width.
2 . The apparatus of claim 1 wherein the circuitry to permute comprises a plurality of configurable switches.
3 . The apparatus of claim 1 wherein the circuitry to permute is to perform a broadcast operation to route a single source data element to each of the plurality of circuitries.
4 . The apparatus of claim 1 wherein to perform the permutation, index values are to be generated by modifying initial index values in accordance with the permutation.
5 . The apparatus of claim 1 wherein the accumulation value having the second bit width is to be stored by an accumulation register.
6 . The apparatus of claim 1 wherein the first bit width is 16 bits and the second bit width is 32 bits.
7 . The apparatus of claim 6 wherein the first and second plurality of source data elements comprise 16-bit floating-point data elements and the accumulation and result values comprise 32-bit floating point data elements.
8 . The apparatus of claim 1 further comprising:
a cache shared by at least some of the plurality of circuitries.
9 . The apparatus of claim 8 wherein the cache comprises a write though cache.
10 . The apparatus of claim 1 wherein the circuitry to permute is to perform the permutation operation based, at least in part, on information included in at least one of the instructions.
11 . The apparatus of claim 10 wherein the circuitry to permute is to perform the permutation operation based, at least in part, on information included in at least one register.