Operation accelerator and compression method
The present disclosure provides example operation accelerators and compression methods. One example operation accelerator includes a storage configured to store first input data, weight data, and a control instruction, and an operation circuit connected to the storage and configured to perform matrix multiplication on the first input data and the weight data, to obtain a computation result. The operation accelerator further includes a compression module configured to compress the computation result to obtain compressed data, as well as a controller connected to the storage and configured to obtain the control instruction from the storage, and when the control instruction includes instructions to compress the computation result, control the compression module to compress the computation result to obtain the compressed data. The operation accelerator further includes a direct memory access controller connected to the compression module and configured to store the compressed data in a memory outside the operation accelerator.
1. An operation accelerator, comprising:
a storage, configured to store first input data, weight data, and a control instruction;
an operation circuit connected to the storage, configured to perform matrix multiplication on the first input data and the weight data, to obtain a computation result, wherein the control instruction instructs whether to compress the computation result;
a compression module, configured to compress the computation result to obtain compressed data;
a controller connected to the storage and configured to:
obtain the control instruction from the storage; and
when the control instruction comprises instructions to compress the computation result, control the compression module to compress the computation result to obtain the compressed data; and
a direct memory access controller (DMAC) connected to the compression module, configured to store the compressed data in a memory outside the operation accelerator.
2. The operation accelerator according to claim 1 , further comprising:
a decompression module connected to the DMAC and the storage, configured to:
receive the compressed data obtained by the DMAC from the memory;
decompress the compressed data; and
store the decompressed data in the storage as second input data; and
wherein the operation circuit is further configured to obtain the second input data from the storage to perform matrix multiplication.
3. The operation accelerator according to claim 2 , wherein the control instruction is used to instruct whether to decompress the computation result; and
the controller is configured to:
parse the control instruction; and
when the control instruction comprises instructions to decompress the computation result, control the decompression module to decompress the obtained compressed data.
4. The operation accelerator according to claim 3 , wherein:
the storage is further configured to store the computation result of the operation circuit; and
the controller is further configured to, when the control instruction skips instructing to compress or decompress the computation result:
control the DMAC to store the computation result in the storage into the memory; and
control the DMAC to store the computation result in the memory into the storage.
5. The operation accelerator according to claim 2 , wherein:
the control instruction is used to instruct whether to decompress the computation result;
the controller is configured to distribute the control instruction to the compression module and the decompression module;
the compression module is configured to:
parse the control instruction; and
when the control instruction comprises instructions to compress the computation result, compress the computation result to obtain the compressed data; and
the decompression module is configured to:
parse the control instruction; and
when the control instruction comprises instructions to decompress the computation result, decompress the obtained compressed data.
6. The operation accelerator according to claim 5 , wherein:
the compression module is further configured to, when the control instruction skips instructing to compress the computation result, control the DMAC to store the computation result in the memory; and
the decompression module is further configured to, when the control instruction skips instructing to decompress the computation result, control the DMAC to store the computation result in the memory into the storage.
7. The operation accelerator according to claim 2 , wherein the compression module comprises a fragmentation module and at least one compression engine, wherein:
the fragmentation module is configured to perform fragmentation processing on the computation result to obtain at least one sub-computation result; and
each compression engine in the at least one compression engine is configured to compress a respective sub-computation result in the at least one sub-computation result to obtain respective sub-compressed data, wherein a sum of all the respective sub-compressed data generated by each compression engine in the at least one compression engine forms the compressed data.
8. The operation accelerator according to claim 7 , wherein each compression engine in the at least one compression engine is configured to:
compress the respective sub-computation result to obtain a respective sub-compression result;
compare size of the respective sub-compression result with size of the respective sub-computation result; and
when the size of the respective sub-compression result is greater than the size of the respective sub-computation result, use the respective sub-computation result as the respective sub-compressed data; or
when the size of the respective sub-compression result is not greater than the size of the respective sub-computation result, use the respective sub-compression result as the respective sub-compressed data.
9. The operation accelerator according to claim 8 , wherein each compression engine in the at least one compression engine is further configured to:
when the size of the respective sub-compression result is greater than the size of the respective sub-computation result:
generate a respective compression failure identifier corresponding to the respective sub-compressed data; and
store the respective compression failure identifier in the memory; and
when the size of the respective sub-compression result is not greater than the size of the respective sub-computation result:
generate a respective compression success identifier corresponding to the respective sub-compressed data; and
store the respective compression success identifier in the memory.
10. The operation accelerator according to claim 9 , wherein the decompression module is configured to:
receive the respective sub-compressed data from the memory; and
when an identifier corresponding to the respective sub-compressed data is the respective compression failure identifier, store the respective sub-compressed data into the storage as the second input data; or
when the identifier corresponding to the respective sub-compressed data is the respective compression success identifier:
decompress, as respective sub-decompressed data, the respective sub-compressed data; and
store the respective sub-decompressed data into the storage as the second input data.
11. A compression method, wherein the compression method is applied to an operation accelerator, the operation accelerator comprises a storage, and the method comprises:
obtaining a computation result by performing matrix multiplication on first input data and weight data obtained from the storage;
obtaining a control instruction, wherein the control instruction instructs whether to compress the computation result; and
when the control instruction comprises instructions to compress the computation result:
obtaining compressed data by compressing the computation result; and
storing the compressed data in a memory outside the operation accelerator.
12. The method according to claim 11 , further comprising:
obtaining the compressed data from the memory;
decompressing the compressed data;
storing the decompressed data into the storage as second input data; and
performing matrix multiplication on the second input data obtained from the storage.
13. The compression method according to claim 12 , wherein the control instruction is used to instruct whether to decompress the computation result, and the method further comprises:
parsing the control instruction; and
wherein the decompressing the compressed data comprises:
when the control instruction comprises instructions to decompress the computation result, decompressing the compressed data.
14. The compression method according to claim 13 , wherein the method further comprises:
when the control instruction skips instructing to compress or decompress the computation result:
storing the computation result in the memory; and
storing the computation result in the memory into the storage.
15. A compression method, wherein the compression method is applied to an operation accelerator, the operation accelerator comprises a storage, and the method comprises:
obtaining a computation result by performing matrix multiplication on first input data and weight data obtained from the storage;
obtaining a control instruction, wherein the control instruction is used to instruct whether to compress and decompress the computation result;
when the control instruction comprises instructions to compress the computation result:
obtaining compressed data by compressing the computation result; and
storing the compressed data in a memory outside the operation accelerator; and
when the control instruction skips instructing to compress the computation result, storing the computation result in the memory outside the operation accelerator.
16. The method according to claim 15 , wherein the method further comprises:
when the control instruction comprises instructions to decompress the computation result:
decompressing the compressed data obtained from the memory; and
performing matrix multiplication using the decompressed data as second input data; and
when the control instruction skips instructing to decompress the computation result, performing matrix multiplication using the computation result obtained from the memory as the second input data.
17. An operation accelerator, comprising:
one or more processors; and
a non-transitory computer readable medium storing a program to be executed by the one or more processors, wherein the program comprises instructions that cause the one or more processors to perform operations comprising:
storing, in a storage, first input data and weight data;
obtaining a computation result by performing matrix multiplication on the first input data and the weight data;
storing, in the storage, a control instruction, wherein the control instruction instructs whether to compress the computation result;
obtaining the control instruction from the storage; and
when the control instruction comprises instructions to compress the computation result:
obtaining compressed data by compressing the computation result; and
storing the compressed data in a memory outside the operation accelerator.
18. The operation accelerator according to claim 17 , wherein the operations further comprise:
receiving the compressed data from the memory;
decompressing the compressed data;
storing the decompressed data in the storage as second input data; and
obtaining the second input data from the storage to perform matrix multiplication.
19. The operation accelerator according to claim 18 , wherein the control instruction is used to instruct whether to decompress the computation result, and the operations further comprises:
parsing the control instruction; and
when the control instruction comprises instructions to decompress the computation result, decompressing the compressed data.
20. The operation accelerator according to claim 19 , wherein the operations further comprise:
storing, in the storage, the computation result; and
when the control instruction skips instructing to compress or decompress the computation result, storing the computation result in the storage into the memory.