Method and apparatus for distributed and cooperative computation in artificial neural networks

1 . An apparatus comprising:

an input/output (I/O) interface to a dynamic random access memory (DRAM);

a processor coupled with the I/O interface, the processor having a plurality of cores and a shared cache to be shared by the plurality of cores;

a scratchpad memory;

a plurality of processing circuits coupled with the I/O interface, the plurality of processing circuits to perform neural network (NN) processing, the plurality of processing circuits to operate at a lower clock frequency than the I/O interface, the plurality of processing circuits including a first processing circuit and a second processing circuit,

the first processing circuit to receive data for neurons and weights associated with the neurons via the I/O interface, wherein the I/O interface is to receive the weights as packed weights, the first processing circuit to process the data for the neurons and the weights associated with the neurons to generate partial results, wherein the data for the neurons and the partial results are to be stored in the scratchpad memory, which is to be shared by the plurality of processing circuits, and

the second processing circuit to process the partial results to generate additional partial results or final results; and

an interconnect coupled between the first and second processing circuits to provide the partial results from the first processing circuit to the second processing circuit.

2 . The apparatus of claim 1 , wherein the second processing circuit is communicatively coupled with the first processing circuit by a unidirectional point-to-point link via the interconnect.

3 . The apparatus of claim 1 , further comprising a plurality of latches to store the partial results prior to the partial results being provided to the second processing circuit.

4 . The apparatus of claim 1 , wherein the first processing circuit comprises a plurality of latches to store the data for the neurons.

5 . The apparatus of claim 1 , wherein the partial results are to be provided to the second processing circuit without being sent through the I/O interface to the DRAM.

6 . The apparatus of claim 1 , wherein the first processing circuit is to generate the partial results which represent only part of a layer of a neural network, and wherein the second processing circuit is to use the partial results to further process the layer.

7 . The apparatus of claim 1 , further comprising a control circuit to control providing the data to the first processing circuit.

8 . The apparatus of claim 1 , wherein the first processing circuit includes multipliers and adders to perform dot product operations.

9 . An apparatus comprising:

an input/output (I/O) interface to a dynamic random access memory (DRAM);

a processor coupled with the I/O interface, the processor having a plurality of cores and a shared cache to be shared by the plurality of cores;

a scratchpad memory;

a plurality of processing circuits coupled with the I/O interface, the plurality of processing circuits to perform neural network (NN) processing, the plurality of processing circuits to operate at a lower clock frequency than the I/O interface, the plurality of processing circuits including a first processing circuit and a second processing circuit,

the first processing circuit to receive data for neurons and weights associated with the neurons via the I/O interface, the first processing circuit to process the data for the neurons and the weights associated with the neurons to generate partial results, wherein the first processing circuit includes multipliers and adders to perform dot product operations, wherein the partial results represent only part of a layer of a neural network, and wherein the data for the neurons and the partial results are to be stored in the scratchpad memory, which is to be shared by the plurality of processing circuits, and which is closer to the plurality of processing circuits than the DRAM, and

the second processing circuit to process the partial results to generate additional partial results or final results, wherein the partial results are to be provided to the second processing circuit without being sent through the I/O interface to the DRAM, and wherein the second processing circuit is to use the partial results to further process the layer; and

an interconnect coupled between the first and second processing circuits to provide the partial results from the first processing circuit to the second processing circuit.

10 . The apparatus of claim 9 , wherein the second processing circuit is communicatively coupled with the first processing circuit by a unidirectional point-to-point link via the interconnect.

11 . The apparatus of claim 9 , wherein the first processing circuit comprises a plurality of latches to store the data for the neurons.

12 . A system comprising:

a system memory;

an input/output (I/O) interface coupled with the system memory;

a processor coupled with the I/O interface, the processor having a plurality of cores and a shared cache to be shared by the plurality of cores;

a scratchpad memory;

a plurality of processing circuits coupled with the I/O interface, the plurality of processing circuits to perform neural network (NN) processing, the plurality of processing circuits to operate at a lower clock frequency than the I/O interface, the plurality of processing circuits including a first processing circuit and a second processing circuit,

the first processing circuit to receive data for neurons and weights associated with the neurons via the I/O interface, the first processing circuit to process the data for the neurons and the weights associated with the neurons to generate partial results, wherein the data for the neurons and the partial results are to be stored in in scratchpad memory, which is to be shared by the plurality of processing circuits, and

the second processing circuit to process the partial results to generate additional partial results or final results; and

an interconnect coupled between the first and second processing circuits to provide the partial results from the first processing circuit to the second processing circuit.

13 . The system of claim 12 , wherein the second processing circuit is communicatively coupled with the first processing circuit by a unidirectional point-to-point link via the interconnect, and further comprising a communication device coupled with the processor.

14 . The system of claim 12 , further comprising circuitry to store the partial results prior to the partial results being provided to the second processing circuit, and further comprising a network controller coupled with the processor.

15 . The system of claim 12 , further comprising a plurality of latches to store the partial results prior to the partial results being provided to the second processing circuit, and further comprising an I/O device coupled with the processor.

16 . The system of claim 12 , wherein the first processing circuit comprises a plurality of latches to store the data for the neurons, and further comprising a network controller coupled with the processor.

17 . The system of claim 12 , wherein the partial results are to be provided to the second processing circuit without being sent through the I/O interface to the system memory, and wherein the system memory comprises a dynamic random access memory (DRAM).

18 . The system of claim 12 , wherein the first processing circuit is to generate the partial results which represent only part of a layer of a neural network, and wherein the second processing circuit is to use the partial results to further process the layer, and further comprising a communication device coupled with the processor.

19 . A method comprising:

exchanging data with a system memory through an input/output (I/O) interface;

processing the data with a plurality of cores of a processor;

storing data from each of the plurality of cores to a shared cache of the processor;

performing neural network (NN) processing with a plurality of processing circuits, including a first processing circuit and a second processing circuit, the NN processing to be performed at a lower clock frequency than a clock frequency used by the I/O interface for the exchanging of the data with the system memory, the NN processing including:

receiving, by the first processing circuit, data for neurons and weights associated with the neurons via the I/O interface,

processing, by the first processing circuit, the data for the neurons and the weights associated with the neurons to generate partial results,

storing the data and the partial results in a scratchpad memory closer to the plurality of processing circuits than the system memory and shared by the plurality of processing circuits, and

processing, by the second processing circuit, the partial results to generate additional partial results or final results; and

providing the partial results from the first processing circuit to the second processing circuit via an interconnect.

20 . The method of claim 19 , wherein providing the partial results from the first processing circuit to the second processing circuit is done without sending the partial results through the I/O interface to the system memory.

21 . The method of claim 19 , wherein providing the partial results from the first processing circuit to the second processing circuit via an interconnect comprises communicating the partial results by a unidirectional point-to-point link via the interconnect.

22 . The apparatus of claim 9 , wherein the I/O interface is to receive the weights as a packed weights.