Architecture for AI accelerator platform

1 . A compute tile, comprising:

a plurality of compute engines, each of the plurality of compute engines including a compute-in-memory (CIM) hardware module, the CIM hardware module including fully addressable memory cells and hardware compute logic coupled with the fully addressable memory cells, the CIM hardware module storing a plurality of weights corresponding to a matrix in the fully addressable memory cells, and the hardware compute logic being configured to perform a vector-matrix multiplication (VMM) for the matrix and including a plurality of logic gates for the fully addressable memory cells, a logic gate of the plurality of logic gates having a first input from a corresponding memory cell of the fully addressable memory cells, a second input for a bit of an input vector, and an output providing a multiplication of a memory bit stored in the corresponding memory cell and the bit of the input vector, wherein each of the plurality of compute engines includes a local update module coupled with the CIM hardware module and configured to locally update at least a portion of the plurality of weights, wherein the local update module resides on the same integrated circuit as the CIM module, wherein the local update module is configured to receive a weight of the plurality of weights from a memory cell of the fully addressable memory cells in the CIM hardware module, locally add a weight update to the weight to provide an updated weight, and write the updated weight to the memory cell; and

a general-purpose (GP) processor coupled with the plurality of compute engines and configured to provide control instructions and data to the plurality of compute engines, the GP processor being a reduced instruction set computer (RISC) processor;

wherein each of the plurality of compute engines is individually coupled with the GP processor such that each of the plurality of compute engines is configured to store data from the GP processor based on a load instruction from the GP processor, perform the VMM, and store output to the GP processor based on a store instruction from the GP processor, wherein at least a first portion of the VMM for each of the plurality of compute engines is performable in parallel with at least a second portion of the VMM for another of the plurality of compute engines, wherein an activation function is applied to an output of at least one compute engine of the plurality of compute engines by the GP processor, wherein the activation function corresponds to a nonlinear operation,

wherein the CIM hardware module of at least one compute engine of the plurality of compute engines is repurposed as a storage memory based on a utilization determination of the at least one compute engine falls below a predefined threshold.

2 . The compute tile of claim 1 , wherein the GP processor is coupled with the plurality of compute engines via a streaming port and a control port, the control port configured to provide the plurality of compute engines with the control instructions and the streaming port being configured to exchange data between the GP processor and each of the plurality of compute engines.

3 . The compute tile of claim 1 , wherein the plurality of compute engines is configured to perform linear operations.

4 . The compute tile of claim 1 , further comprising:

local memory coupled with the GP processor.

5 . The compute tile of claim 4 , further comprising:

a direct memory access (DMA) unit coupled with the local memory and the GP processor, the DMA unit being configured for data transfers to and from the compute tile.

6 . The compute tile of claim 1 , wherein the GP processor further includes: a single instruction multiple data arithmetic logic unit.

7 . The compute tile of claim 1 , wherein the compute tile is included in a plurality of compute tiles.

8 . A system, comprising:

a plurality of compute tiles, each of the plurality of compute tiles including a plurality of compute engines and a general-purpose (GP) processor coupled with the plurality of compute engines, each of the plurality of compute engines including a compute-in-memory (CIM) hardware module, the CIM hardware module including fully addressable memory cells and hardware compute logic coupled with the fully addressable memory cells, the CIM hardware module storing a plurality of weights corresponding to a matrix in the fully addressable memory cells, and the hardware compute logic being configured to perform a vector-matrix multiplication (VMM) for the matrix and including a plurality of logic gates for the fully addressable memory cells, a logic gate of the plurality of logic gates having a first input from a corresponding memory cell of the fully addressable memory cells, a second input for a bit of an input vector, and an output providing a multiplication of a memory bit stored in the corresponding memory cell and the bit of the input vector, the GP processor being configured to provide control instructions and data to the plurality of compute engines, the GP processor being a reduced instruction set computer (RISC) processor, wherein each of the plurality of compute engines includes a local update module coupled with the CIM hardware module and configured to locally update at least a portion of the plurality of weights, wherein the local update module resides on the same integrated circuit as the CIM module, wherein the local update module is configured to receive a weight of the plurality of weights from a memory cell of the fully addressable memory cells in the CIM hardware module, locally add a weight update to the weight to provide an updated weight, and write the updated weight to the memory cell;

wherein each of the plurality of compute engines is individual coupled with the GP processor such that each of the plurality of compute engines is configured to store data from the GP processor based on a load instruction from the GP processor, perform the VMM, and store output to the GP processor based on a store instruction from the GP processor, wherein at least a first portion of the VMM for each of the plurality of compute engines is performable in parallel with at least a second portion of the VMM for another of the plurality of compute engines, wherein an activation function is applied to an output of at least one compute engine of the plurality of compute engines by the GP processor, wherein the activation function corresponds to a nonlinear operation,

wherein the CIM hardware module of at least one compute engine of the plurality of compute engines is repurposed as a storage memory based on a utilization determination of the at least one compute engine falls below a predefined threshold.

9 . The system of claim 8 , wherein the GP processor is coupled with the plurality of compute engines via a streaming port and a control port, the control port configured to provide the plurality of compute engines with the control instructions and the streaming port being configured to exchange data between the GP processor and each of the plurality of compute engines.

10 . The system of claim 8 , wherein the plurality of compute engines is configured to perform linear operations.

11 . The system of claim 8 , wherein at least one of the plurality of compute tiles further includes local memory coupled with the GP processor.

12 . The system of claim 11 , wherein the at least one of the plurality of compute tiles further includes:

at least one of a direct memory access (DMA) unit coupled with the local memory and the GP processor for the at least one of the plurality of compute tiles, the DMA unit being configured for data transfers to and from the compute tile or a data bus configured to transfer data to or from the local memory.

13 . The system of claim 8 , wherein the GP processor further includes: a single instruction multiple data arithmetic logic unit.

14 . A method, comprising:

providing, to at least one compute engine of a plurality of compute engines, an input vector, the at least one compute engine storing a plurality of weights corresponding to a matrix, each of the plurality of compute engines including a compute-in-memory (CIM) hardware module, the CIM hardware module including fully addressable memory cells and hardware compute logic coupled with the fully addressable memory cells, the CIM hardware module of the at least one compute engine storing the plurality of weights in the fully addressable memory cells, and the hardware compute logic being configured to perform a vector-matrix multiplication (VMM) for the matrix and including a plurality of logic gates for the fully addressable memory cells, a logic gate of the plurality of logic gates having a first input from a corresponding memory cell of the fully addressable memory cells, a second input for a bit of the input vector, and an output providing a multiplication of a memory bit stored in the corresponding memory cell and the bit of the input vector, the at least one compute engine performing a VMM between the input vector and the matrix to provide an output, wherein each of the plurality of compute engines includes a local update module coupled with the CIM hardware module and configured to locally update at least a portion of the plurality of weights, wherein the local update module resides on the same integrated circuit as the CIM module, wherein the local update module is configured to receive a weight of the plurality of weights from a memory cell of the fully addressable memory cells in the CIM hardware module, locally add a weight update to the weight to provide an updated weight, and write the updated weight to the memory cell;

applying, by a general-purpose (GP) processor coupled with the plurality of compute engines, a function to the output, the GP processor being configured to provide control instructions and data to the plurality of compute engines, the GP processor being a reduced instruction set computer (RISC) processor;

wherein each of the plurality of compute engines is coupled with the GP processor such that each of the plurality of compute engines is configured to store data from the GP processor based on a load instruction from the GP processor, perform the VMM, and store output to the GP processor based on a store instruction from the GP processor, wherein at least a first portion of the VMM for each of the plurality of compute engines is performable in parallel with at least a second portion of the VMM for another of the plurality of compute engines, wherein an activation function is applied to an output of at least one compute engine of the plurality of compute engines by the GP processor, wherein the activation function corresponds to a nonlinear operation,

wherein the CIM hardware module of at least one compute engine of the plurality of compute engines is repurposed as a storage memory based on a utilization determination of the at least one compute engine falls below a predefined threshold.

15 . The method of claim 14 , further comprising:

storing, in the at least one compute engine, the plurality of weights.

16 . The method of claim 14 , wherein the GP processor is coupled with the plurality of compute engines via a streaming port and a control port, the control port configured to provide the plurality of compute engines with the control instructions and the streaming port being configured to exchange data between the GP processor and each of the plurality of compute engines.

17 . The method of claim 15 , wherein the plurality of compute engines is configured to perform linear operations.

18 . The compute tile of claim 1 , further comprising:

at least one connector coupling the plurality of compute engines with the GP processor by at least one connector such that each of the plurality of compute engines receives data from and provides output to the GP processor through the at least one connector.

19 . The compute tile of claim 1 , wherein each of the plurality of compute engines is coupled with the GP processor such that data is received by each of the plurality of compute engines only through the GP processor.

20 . The compute tile of claim 1 , wherein the plurality of weights stored in the CIM hardware module comprises at least one duplicated weight.