Multi-core systolic processor system for neural network processing
At least a subset of first processing units of a first arrangement of a first systolic processing chip is assigned to a first layer of a neural network and at least a subset of second processing units of a second arrangement of the first systolic processing chip is assigned to a second layer of the neural network. At least a subset of third processing units of a third arrangement of a second systolic processing chip is assigned to a third layer of the neural network. Input data is processed using the subset of the first processing units to generate first activation output values. The first activation output values are systollically pulsed to the subset of the second processing units and processed to generate second activation output values. The second activation output values are processed using the subset of the third processing units of the second systolic processing chip.
1. A device for performing computations of a neural network comprising at least first, second, and third layers, the device comprising:
a first systolic processing chip including at least:
a first arrangement of first processing units, and
a second arrangement of second processing units, wherein the first arrangement of first processing units is configured to systolically pulse data to the second arrangement of second processing units;
a second systolic processing chip including at least a third arrangement of third processing units, wherein the second systolic processing chip is configured to receive data output from the second arrangement of first systolic processing chip and provide the data to the third arrangement of processing units; and
circuitry configured to:
assign at least a subset of the first processing units of the first arrangement to perform computations of particular nodes of the first layer of the neural network, wherein each assigned first processing unit is identified by a respective address;
assign at least a subset of the second processing units of the second arrangement to perform computations of particular nodes of the second layer of the neural network; and
assign at least a subset of the third processing units of the third arrangement to perform computations of particular nodes of the third layer of the neural network.
2. The device of claim 1 , wherein each of the first and second arrangements of processing units comprise a column of processing units embedded in the first systolic processing chip.
3. The device of claim 1 , wherein:
the at least a subset of the first processing units is configured to:
process input values of the neural network to generate first activation output values; and
systolically pulse the first activation output values to the at least a subset of the second processing units; and
the at least a subset of the second processing units is configured to process the first activation output values to generate second activation output values; and
the circuitry is further configured to transmit the second activation output values to the third arrangement of third processing units.
4. The device of claim 3 , wherein the first systolic processing chip is configured to systolically pulse the second activation output values through the second arrangement of second processing units to a memory.
5. The device of claim 4 , wherein the circuitry is further configured to retrieve the second activation outputs from the memory and provide the second activation outputs to the second systolic processing chip.
6. The device of claim 1 , wherein the computations of a particular one of the second processing units includes multiplication of an activation output received from one of the first processing units with corresponding weights.
7. The device of claim 6 , wherein one or more of the corresponding weights are stored locally at the one of the second processing units.
8. The device of claim 6 , wherein one or more of the corresponding weights are retrieved from a memory external to the one of the second processing units.
9. The device of claim 1 , wherein the circuitry is further configured to:
initially configure the first and second systolic processing chips to perform forward propagation according to the neural network, and
subsequently configure the first and second systolic processing chips to back propagate deviations from expected outputs.
10. The device of claim 9 , wherein the circuitry is further configured to update weights of the neural network based on the back propagation.
11. A method for performing computations of a neural network comprising at least first, second, and third layers, the method comprising:
assigning at least a subset of first processing units of a first arrangement of a first systolic processing chip to the first layer of the neural network, wherein each assigned first processing unit is identified by a respective address;
assigning at least a subset of second processing units of a second arrangement of the first systolic processing chip to the second layer of the neural network;
assigning at least a subset of third processing units of a third arrangement of a second systolic processing chip to the third layer of the neural network;
receiving input data of the neural network;
processing the input data using the at least a subset of the first processing units to generate first activation output values;
systolically pulsing the first activation output values to the at least a subset of the second processing units;
processing the first activation output values using the at least a subset of the second processing units to generate second activation output values; and
processing the second activation output values using the at least a subset of the third processing units of the second systolic processing chip.
12. The method of claim 11 , further comprising systolically pulsing the second activation output values through the second processing units of the second arrangement to a memory.
13. The method of claim 12 , further comprising:
retrieving the second activation outputs from the memory; and
providing the second activation outputs to the second systolic processing chip.
14. The method of claim 11 , wherein processing the first activation output values comprises, via one of the second processing units, multiplying at least one of the first activation output values with a weight.
15. The method of claim 14 , further comprising retrieving the weight from a local memory of the one of the second processing units.
16. The method of claim 14 , further comprising retrieving the weight from a memory external to the one of the second processing units.
17. A non-transitory computer storage medium storing computer-executable instructions that, when executed by a processor, cause the processor to perform the following method:
assigning at least a subset of first processing units of a first arrangement of a first systolic processing chip to a first layer of a neural network including at least the first layer and second and third layers, wherein each assigned first processing unit is identified by a respective address;
assigning at least a subset of second processing units of a second arrangement of the first systolic processing chip to the second layer of the neural network;
assigning at least a subset of third processing units of a third arrangement of a second systolic processing chip to the third layer of the neural network;
receiving input data of the neural network;
processing the input data using the at least a subset of the first processing units to generate first activation output values;
systolically pulsing the first activation output values to the at least a subset of the second processing units;
processing the first activation outputs using the at least a subset of the second processing units to generate second activation output values; and
processing the second activation output values using the at least a subset of the third processing units of the second systolic processing chip.
18. The non-transitory computer storage medium of claim 17 , wherein the method further comprises systolically pulsing the second activation output values through the second processing units of the second arrangement to a memory.
19. The non-transitory computer storage medium of claim 18 , wherein processing the first activation output values comprises, via one of the second processing units, multiplying at least one of the first activation output values with a weight.
20. The non-transitory computer storage medium of claim 19 , wherein the method further comprises retrieving the weight from a local memory of the one of the second processing units.