Register-based matrix multiplication with multiple matrices per register
Techniques for performing matrix multiplication in a data processing apparatus are disclosed, comprising apparatuses, matrix multiply instructions, methods of operating the apparatuses, and virtual machine implementations. Registers, each register for storing at least four data elements, are referenced by a matrix multiply instruction and in response to the matrix multiply instruction a matrix multiply operation is carried out. First and second matrices of data elements are extracted from first and second source registers, and plural dot product operations, acting on respective rows of the first matrix and respective columns of the second matrix are performed to generate a square matrix of result data elements, which is applied to a destination register. A higher computation density for a given number of register operands is achieved with respect to vector-by-element techniques.
1. Apparatus comprising:
register storage circuitry having a plurality of registers, each register to store at least four data elements;
decoder circuitry responsive to a matrix multiply instruction to generate control signals, wherein the matrix multiply instruction specifies in the plurality of registers: a first source register, a second source register, and a destination register; and
data processing circuitry responsive to the control signals to perform a matrix multiply operation comprising:
extracting a first matrix of data elements from the first source register;
extracting a second matrix of data elements from the second source register;
performing plural dot product operations to generate a square matrix of result data elements, wherein each dot product operation acts on a respective row of the first matrix of data elements and a respective column of the second matrix of data elements to generate a respective result data element of the square matrix of result data elements; and
applying the square matrix of result data elements to the destination register,
wherein:
the matrix multiply operation is a first matrix multiply operation and the square matrix of result data elements is a first square matrix of result data elements, wherein the destination register has a storage capacity accommodating at least eight data elements and to store at least a second square matrix of result data elements, wherein the data processing circuitry is responsive to the control signals to apply the first square matrix of result data elements to a first portion of the destination register, and wherein the data processing circuitry is further responsive to the control signals to perform a second matrix multiply operation to generate the second square matrix of result data elements and to apply the second square matrix of result data elements to a second portion of the destination register;
the data processing circuitry is responsive to the control signals to perform the first matrix multiply operation comprising:
extracting the first matrix of data elements from a first portion of the first source register;
extracting the second matrix of data elements from a first portion of the second source register; and
applying the first square matrix of result data elements to the first portion of the destination register; and
the data processing circuitry is responsive to the control signals to perform the second matrix multiply operation comprising:
extracting the first matrix of data elements from a second portion of the first source register;
extracting the second matrix of data elements from a second portion of the second source register; and
applying the second square matrix of result data elements to the second portion of the destination register.
2. The apparatus of claim 1 , wherein applying the first square matrix of result data elements to the destination register comprises storing the first square matrix of result data elements in the destination register.
3. The apparatus of claim 1 , wherein applying the first square matrix of result data elements to the destination register comprises accumulating the first square matrix of result data elements with a previously stored square matrix of result data elements stored in the destination register.
4. The apparatus of claim 1 , wherein each data element of the first matrix of data elements and each data element of the second matrix of data elements has a source element bit length which is equal to a result element bit length of each data element of the first square matrix of result data elements.
5. The apparatus of claim 1 , wherein each data element of the first matrix of data elements and each data element of the second matrix of data elements has a source element bit length which is different from a result element bit length of each data element of the first square matrix of result data elements.
6. The apparatus of claim 1 , wherein the matrix multiply instruction specifies a dimension of the first square matrix of result data elements.
7. The apparatus of claim 6 , wherein a storage capacity of the destination register is capable of storing more than one square matrix of result data elements having the dimension specified in the matrix multiply instruction.
8. The apparatus of claim 1 , wherein the data elements of the first matrix of data elements, the data elements of the second matrix of data elements, and the result data elements are floating point values.
9. The apparatus of claim 1 , wherein the data elements of the first matrix of data elements, the data elements of the second matrix of data elements, and the result data elements are integer values.
10. The apparatus any claim 1 , wherein the first matrix of data elements and the second matrix of data elements are non-square rectangular matrices.
11. The apparatus of claim 1 , wherein the decoder circuitry is further responsive to a configuration instruction to generate configuration control signals, wherein the configuration instruction specifies a maximum vector length, and the data processing circuitry is responsive to the configuration control signals to constrain subsequent matrix multiply operations such that a total size of the result data elements does not exceed the maximum vector length.
12. The apparatus of claim 1 , wherein the data processing circuitry is further responsive to the control signals to perform at least one further matrix multiply operation to generate at least a further square matrix of result data elements.
13. The apparatus of claim 1 , wherein the decoder circuitry is responsive to a copying load instruction preceding the matrix multiply instruction and specifying a memory location and the first source register to generate copying load control signals, and the data processing circuitry is responsive to the copying load control signals to perform a copying load operation comprising:
loading the first matrix of data elements from the specified memory location into the first portion of the first source register; and
copying the first matrix of data elements into the second portion of the first source register.
14. The apparatus of claim 1 , wherein the matrix multiply instruction further specifies at least one of:
a third source register, and wherein the matrix multiply operation comprises extracting the first matrix of data elements from the first source register and the third source register;
a fourth source register, and wherein the matrix multiply operation comprises extracting the second matrix of data elements from the second source register and the fourth source register; and
a further destination register, and wherein the matrix multiply operation comprises applying the first square matrix of result data elements split between the destination register and the further destination register.
15. A non-transitory, computer-readable storage medium storing a virtual machine computer program comprising program instructions to control a host data processing apparatus to provide an instruction execution environment corresponding to the apparatus according to claim 1 .
16. A method of operating a data processing apparatus comprising:
generating, by decoder circuitry, control signals in response to a matrix multiply instruction, wherein the matrix multiply instruction specifies in a plurality of registers, each register capable of storing at least four data elements: a first source register, a second source register, and a destination register; and
performing, by data processing circuitry, a matrix multiply operation in response to the control signals comprising:
extracting a first matrix of data elements from the first source register;
extracting a second matrix of data elements from the second source register;
performing plural dot product operations to generate a square matrix of result data elements, wherein each dot product operation acts on a respective row of the first matrix of data elements and a respective column of the second matrix of data elements to generate a respective result data element of the square matrix of result data elements; and
applying the square matrix of result data elements to the destination register,
wherein:
the matrix multiply operation is a first matrix multiply operation and the square matrix of result data elements is a first square matrix of result data elements, wherein the destination register has a storage capacity accommodating at least eight data elements and to store at least a second square matrix of result data elements, wherein the matrix multiply operation comprises applying the first square matrix of result data elements to a first portion of the destination register;
the method further comprises responding to the control signals to perform a second matrix multiply operation to generate the second square matrix of result data elements and applying the second square matrix of result data elements to a second portion of the destination register;
the first matrix multiply operation comprises:
extracting the first matrix of data elements from a first portion of the first source register;
extracting the second matrix of data elements from a first portion of the second source register; and
applying the first square matrix of result data elements to the first portion of the destination register; and
the second matrix multiply operation comprises:
extracting the first matrix of data elements from a second portion of the first source register;
extracting the second matrix of data elements from a second portion of the second source register; and
applying the second square matrix of result data elements to the second portion of the destination register.
17. A non-transitory, computer-readable storage medium storing a virtual machine computer program comprising program instructions to control a host data processing apparatus to provide an instruction execution environment in which the method of claim 16 can be carried out.