Using shared data bus to support systolic array tiling
To improve utilization of a systolic array, each row of the array is provided with a number of row input data buses. Each of the row input data buses can be operable to transfer either feature map (FMAP) input elements or weight values into the processing elements of the corresponding row of the array. By using such row input data buses, concurrent matrix multiplications as well as faster background loading can be achieved in the array.
1 . A neural network processor comprising:
an array of processing elements arranged in rows and columns, the array of processing elements operable to perform multiple matrix multiplications,
wherein each processing element (PE) is configured to multiply a feature map (FMAP) input element with a weight value to generate a multiplication result, add the multiplication result to a column input to generate a partial sum, and output the partial sum to a column output, and
wherein each row of the array includes a set of row input data buses, and the set of row input data buses includes a plurality of row input data buses operable to transfer FMAP input elements to the corresponding row or to transfer weight values to the corresponding row.
2 . The neural network processor of claim 1 , wherein the array of processing elements is operable to perform multiple matrix multiplications concurrently using subarrays of the array.
3 . The neural network processor of claim 1 , wherein the plurality of row input data buses includes four row input data buses operable to transfer the weight values in parallel.
4 . The neural network processor of claim 1 , wherein the plurality of row input data buses includes four row input data buses operable to transfer the FMAP input elements in parallel.
5 . An integrated circuit device comprising:
an array of processing elements,
wherein each row of the array includes multiple row input data buses, and
wherein each processing element in a row of the array includes:
a multiplier to multiply a first operand with a second operand;
a first multiplexer to select the first operand from the multiple row input data buses of the row; and
an adder to add an output of the multiplier to a column input of the processing element to generate a column output of the processing element.
6 . The integrated circuit device of claim 5 , wherein the multiple row input data buses of each row include four row input data buses operable to transfer a plurality of first operands in parallel.
7 . The integrated circuit device of claim 6 , wherein the first operand is a feature map (FMAP) input element.
8 . The integrated circuit device of claim 6 , wherein the first operand is a weight value.
9 . The integrated circuit device of claim 6 , wherein the multiple row input data buses of each row further include an additional row input data bus operable to transfer the second operand.
10 . The integrated circuit device of claim 5 , wherein the array is operable to perform concurrent matrix multiplications each using a subarray of the array.
11 . The integrated circuit device of claim 5 , wherein the array is operable to perform a matrix multiplication using an entirety of the array.
12 . The integrated circuit device of claim 5 , wherein the array is operable to switch between performing a matrix multiplication using an entirety of the array, and performing concurrent matrix multiplications using subarrays of the array.
13 . The integrated circuit device of claim 5 , further comprising:
a second multiplexer to select the second operand from the multiple row input data buses of the row.
14 . A method comprising:
receiving, by a processing element in an array of processing elements, a first operand on a row input data bus that is part of a set of row input data buses for the processing element;
receiving, by the processing element, a plurality of second operands in parallel on a plurality of row input data buses in the set of row input data buses for the processing element;
selecting, by the processing element, a second operand from the plurality of second operands;
multiplying, by the processing element, the first operand with the selected second operand to generate a multiplication result; and
adding, by the processing element, the multiplication result to a column input of the processing element to generate a column output of the processing element.
15 . The method of claim 14 , wherein the set of row input data buses includes four row input data buses to transfer the plurality of second operands in parallel.
16 . The method of claim 15 , wherein the selected second operand is a feature map (FMAP) input element.
17 . The method of claim 15 , wherein the selected second operand is a weight value.
18 . The method of claim 14 , further comprising storing the first operand in a register of the processing element.
19 . The method of claim 14 , wherein the column output is generated as part of a matrix multiplication operation being performed using an entirety of the array.
20 . The method of claim 14 , wherein the column output is generated as part of a matrix multiplication operation being performed concurrently with other matrix multiplication operations using subarrays of the array.