Matrix multiplication in a dynamically spatially and dynamically temporally dividable architecture
A data processing apparatus includes first vector registers and second vector registers, both dynamically spatially and dynamically temporally dividable. Decode circuitry receives one or more matrix multiplication instructions that indicate a set of first elements in the first vector registers and a set of second elements in the second vector registers, and in response to receiving the matrix multiplication instructions they generate a matrix multiplication operation. The matrix multiplication operation causes one or more execution units to perform a matrix multiplication of the set of first elements by the set of second elements and an average bit width of the first elements is different to an average bit width of the second elements.
1 . A data processing apparatus comprising:
first vector registers and second vector registers, both configured to be dynamically spatially and dynamically temporally divided; and
decode circuitry configured to receive one or more matrix multiplication instructions comprising an indication of a set of first elements in the first vector registers and a set of second elements in the second vector registers, and in response to receiving the matrix multiplication instructions to generate a matrix multiplication operation, wherein
the matrix multiplication operation is configured to cause one or more execution units to perform a matrix multiplication of the set of first elements by the set of second elements; and
an average bit width of the first elements is different from an average bit width of the second elements,
wherein:
the matrix multiplication instructions comprise a compressed matrix multiplication instruction that comprises an indication of compression data;
the first elements comprise a single row of n activations, where n is an integer;
the second elements comprise m groups of n compressed weights, where m is an integer greater than 1;
the compression data indicates how the n compressed weights are decompressed to form mn uncompressed weights.
2 . The data processing apparatus according to claim 1 , wherein
the matrix multiplication instructions comprise an indication of a further result register;
the result register is configured to store the first set of bits of the result of the matrix multiplication; and
the further result register is configured to store the second set of bits of the result of the matrix multiplication.
3 . The data processing apparatus according to claim 1 , wherein
the matrix multiplication multiplies fewer rows of the first set of elements than a number of columns of the second set of elements.
4 . The data processing apparatus according to claim 1 , wherein
the matrix multiplication is of one row of the first set of elements and two columns of the second set of elements.
5 . The data processing apparatus according to claim 1 , wherein the bit width of the second elements is four bits or less.
6 . The data processing apparatus according to claim 1 , wherein the bit width of the second elements is one bit.
7 . The data processing apparatus according to claim 6 , wherein
the one or more matrix multiplication instructions comprise an indicator value, or the data processing apparatus comprises a selection register configured to store the indicator value; and
the indicator value is configured to indicate a subset of the weights that are used in the matrix multiplication during a particular beat of the data processing apparatus.
8 . The data processing apparatus according to claim 6 , wherein
bits of at least one of the indication of the set of first elements in the first vector registers and the set of second elements in the second vector registers are used to indicate a subset of the weights that are used in the matrix multiplication during a particular beat of the data processing apparatus.
9 . The data processing apparatus according to claim 1 , wherein the second elements are signed.
10 . The data processing apparatus according to claim 1 , wherein the weights are extended prior to the matrix multiplication.
11 . The data processing apparatus according to claim 1 , wherein
the compression data comprises a plurality of portions, each applicable to one of a plurality of matrix multiplication instructions including the compressed matrix multiplication instruction; and
the compressed matrix multiplication instruction comprises a compression data selector configured to select one of the portions of the compression data to be applied to form the n uncompressed weights.
12 . The data processing apparatus according to claim 11 , wherein
the compression data is applicable to a plurality of the matrix multiplication instructions;
at least some of the matrix multiplication instructions indicate different second elements from each other; and
the compression data comprises a number of items.
13 . The data processing apparatus according to claim 12 , wherein
the compression data is applicable to more than m groups of n weights.
14 . The data processing apparatus according to claim 12 , wherein
the items are ordered in the compression data according to a beat in which they are used within the plurality of matrix multiplication operations.
15 . The data processing apparatus according to claim 12 , wherein
the items are ordered in the compression data such that items used in a same beat of a same single matrix multiplication operation are adjacent.
16 . The data processing apparatus according to claim 11 , wherein
the compression data selector is least significant bits of another parameter of the compressed matrix multiplication instruction.
17 . The data processing apparatus according to claim 11 , wherein
the compression data selector is least significant bits of an address of the second elements.
18 . The data processing apparatus according to claim 17 , wherein
the compression data selector is combinable with a stub address of the first elements to form an address of the first elements; and
the compression data selector is combinable with a stub address of a result register to form an address of result register into which at least a part of the result of the matrix multiplication is stored.
19 . The data processing apparatus according to claim 11 , comprising:
multiplexer circuitry configured to select from between the activations to match with the uncompressed weights that are non-zero to provide as an input to the matrix multiplication.
20 . The data processing apparatus according to claim 19 , wherein
the multiplexer circuitry is configured to select from between a subset of the activations to match with the uncompressed weights that are non-zero.
21 . A data processing apparatus comprising:
first vector registers and second vector registers, both configured to be dynamically spatially and dynamically temporally divided; and
decode circuitry configured to receive one or more matrix multiplication instructions comprising an indication of a set of first elements in the first vector registers and a set of second elements in the second vector registers, and in response to receiving the matrix multiplication instructions to generate a matrix multiplication operation, wherein
the matrix multiplication operation is configured to cause one or more execution units to perform a matrix multiplication of the set of first elements by the set of second elements; and
an average bit width of the first elements is different from an average bit width of the second elements, wherein
the matrix multiplication instructions comprise an uncompressed matrix multiplication instruction;
the first elements comprise a single group of n activations, where n is an integer;
the second elements comprise m groups of n weights, where m is an integer greater than 1; and
a bit width of the second elements is 1/m times a bit width of the first elements.
22 . A non-transitory, computer readable medium containing a computer program for controlling a host data processing apparatus to provide an instruction execution environment comprising:
first data structures and second data structures, both configured to be dynamically spatially and dynamically temporally divided; and
decode logic configured to receive one or more matrix multiplication instructions comprising an indication of a set of first elements in the first data structures and a set of second elements in the second data structures, and in response to receiving the matrix multiplication instructions to generate a matrix multiplication operation, wherein
the matrix multiplication operation is configured to cause execution logic to perform a matrix multiplication of the set of first elements by the set of second elements; and
an average bit width of the first elements is different from an average bit width of the second elements,
wherein:
the matrix multiplication instructions comprise a compressed matrix multiplication instruction that comprises an indication of compression data;
the first elements comprise a single row of n activations, where n is an integer;
the second elements comprise m groups of n compressed weights, where m is an integer greater than 1;
the compression data indicates how the n compressed weights are decompressed to form mn uncompressed weights.