Operation accelerator, processing method, and related device
An operation accelerator, a processing method, and a related device, the operation accelerator including a first memory configured to store an input dataset, a matrix converter configured to perform reading M row vectors from the input dataset, generating a first instruction, and sending the M row vectors and the first instruction to a second memory configured to perform, according to the first instruction, preprocessing on the M row vectors to obtain n row vectors, and storing the n row vectors, where the n row vectors include the M row vectors and (n−M) padding row vectors, the n row vectors are N row vectors in a target matrix, and a storage sequence of the n row vectors in the second memory is consistent with a sequence of the N row vectors in the target matrix.
1. An operation accelerator, comprising:
a matrix converter;
a first memory; and
a second memory coupled to the matrix converter;
wherein the first memory is configured to store an input dataset, the input dataset comprises S row vectors, each of the S row vectors comprises C0 pieces of data, and the C0 pieces of data correspond to an element Aij in an input feature map that is on C0 consecutive channels in a C direction, wherein a size of the input feature map is W*H*C, W is a width of the input feature map, H is a height of the input feature map, and Cis a quantity of channels of the input feature map, Aij is an element in an ith row and a jth column on a W*H plane of the input feature map, i is a positive integer less than or equal to W, and j is a positive integer less than or equal to H;
wherein the matrix converter is configured to:
read M row vectors from the input dataset, wherein the M row vectors are M row vectors in a target matrix, a size of the target matrix is N*C0, N row vectors in the target matrix correspond to N elements Aij, when a convolution kernel continuously slides on the W*H plane of the input feature map for N times, that correspond to an element Wmn in the convolution kernel and that are on the W*H plane, and Wmn is an element in an mth row and an nth column in the convolution kernel, and wherein M is an integer less than or equal to S, N is an integer greater than or equal to M, and C0 is an integer less than or equal to C;
generate a first instruction, wherein the first instruction comprises an instruction for performing preprocessing on the M row vectors, and the preprocessing comprises one or more of sliding, concatenation, or padding; and
send the M row vectors and the first instruction to the second memory; and
wherein the second memory is configured to:
perform, according to the first instruction, the preprocessing on the M row vectors to obtain n row vectors; and
store the n row vectors, wherein the n row vectors comprise the M row vectors and (n−M) padding row vectors, the n row vectors are the N row vectors in the target matrix, and a storage sequence of the n row vectors in the second memory is consistent with a sequence of the N row vectors in the target matrix.
2. The operation accelerator according to claim 1 , wherein N is greater than M, and the N row vectors in the target matrix correspond to N elements Aij, when the convolution kernel continuously slides on a padded W*H plane of the input feature map for N times, that correspond to the element Wmn in the convolution kernel and that are on the padded W*H plane; and
wherein the second memory is further configured to:
pad the (n−M) padding row vectors with fixed values; and
perform, according to the first instruction, a displacement or concatenation operation on the M row vectors and the (n−M) padding row vectors to obtain the n row vectors, and store the n row vectors.
3. The operation accelerator according to claim 1 , further comprising:
a third memory coupled to the matrix converter; and
an operation circuit coupled to the third memory;
wherein the matrix converter is further configured to:
send, to the third memory for P times, the n row vectors stored in the second memory and a corresponding second instruction generated by the matrix converter, wherein P is an integer greater than or equal to 1, and the second instruction comprises a write enable, a write address, and a write size;
wherein the third memory is configured to:
store P*n row vectors according to the corresponding second instruction; and
send the P*n row vectors to the operation circuit, wherein a storage sequence, in the third memory, of n row vectors that are received at each time in the n row vectors received for the P times is consistent with a sequence of the corresponding N row vectors in the target matrix, and storage sequences, in the third memory, of the n row vectors that are received for P times are consistent with receiving sequences of the n row vectors that are received for P times; and
wherein the operation circuit is configured to perform matrix operations on the P*n row vectors.
4. The operation accelerator according to claim 1 , wherein the S row vectors are data corresponding to consecutive S pieces of Aij on the C0 consecutive channels in the C direction, and the S pieces of Aij are first in a W direction and then in an H direction in the input feature map.
5. The operation accelerator according to claim 1 , wherein the matrix converter is further configured to:
read the M row vectors from the S row vectors according to a read instruction, wherein the read instruction comprises a read enable, a read address, a read stride, and a read size, and the read stride is equal to a sliding stride of the convolution kernel.
6. The operation accelerator according to claim 5 , wherein the matrix converter is further configured to:
perform, based on the n row vectors stored in the second memory and the first instruction, a reverse operation on the n row vectors to obtain the M row vectors through restoration, wherein the reverse operation comprises one or more of sliding, splitting, or de-padding; and
restore locations of the M row vectors in the S row vectors according to the read instruction for reading the M row vectors from the input dataset.
7. The operation accelerator according to claim 1 , wherein the preprocessing comprises sliding.
8. The operation accelerator according to claim 1 , wherein the preprocessing comprises padding.
9. The operation accelerator according to claim 1 , wherein the preprocessing comprises concatenation.
10. The operation accelerator according to claim 1 , wherein the preprocessing comprises sliding and concatenation.
11. A method, comprising:
storing an input dataset in a first memory of an operation accelerator, wherein the input dataset comprises S row vectors, each of the S row vectors comprises C0 pieces of data, and the C0 pieces of data correspond to an element Aij in an input feature map that is on C0 consecutive channels in a C direction, wherein a size of the input feature map is W*H*C, W is a width of the input feature map, H is a height of the input feature map, and C is a quantity of channels of the input feature map, Aij is an element in an ith row and a jth column on a W*H plane of the input feature map, i is a positive integer less than or equal to W, and j is a positive integer less than or equal to H;
reading, by a matrix converter of the operation accelerator, M row vectors from the input dataset, wherein the M row vectors are in a target matrix, a size of the target matrix is N*C0, N row vectors in the target matrix correspond to N elements Aij, when a convolution kernel continuously slides on the W*H plane of the input feature map for N times, that correspond to an element Wmn in the convolution kernel and that are on the W*H plane, and Wmn is an element in an mth row and an nth column in the convolution kernel, and wherein M is an integer less than or equal to S, N is an integer greater than or equal to M, and C0 is an integer less than or equal to C;
generating a first instruction by the matrix converter, wherein the first instruction comprises an instruction for performing preprocessing on the M row vectors, and the preprocessing comprises one or more of sliding, concatenation, or padding;
sending, by the matrix converter, the M row vectors and the first instruction to a second memory of the operation accelerator;
performing preprocessing on the M row vectors, by the second memory, to obtain n row vectors; and
storing the n row vectors, by the second memory, wherein the n row vectors comprise the M row vectors and (n−M) padding row vectors, the n row vectors are the N row vectors in the target matrix, and a storage sequence of the n row vectors is consistent with a sequence of the N row vectors in the target matrix.
12. The method according to claim 11 , wherein when N is greater than M, the N row vectors in the target matrix are row vectors corresponding to N elements Aij, when the convolution kernel continuously slides on a padded W*H plane of the input feature map for N times, that correspond to the element Wmn in the convolution kernel and that are on the padded W*H plane; and
wherein performing preprocessing on the M row vectors, by the second memory, to obtain the n row vectors comprises:
padding the (n−M) padding row vectors with fixed values; and
obtaining the n row vectors after the sliding or concatenation is performed on the M row vectors and the (n−M) padding row vectors.
13. The method according to claim 11 , wherein the S row vectors are data corresponding to consecutive S pieces of Aij on the C0 consecutive channels in the C direction, and the S pieces of Aij are first in a W direction and then in an H direction in the input feature map.
14. The method according to claim 11 , wherein reading by the matrix converter of the operation accelerator, the M row vectors from the input dataset comprises:
reading the M row vectors from the S row vectors according to a read instruction, wherein the read instruction comprises a read enable, a read address, a read stride, and a read size, and the read stride is equal to a sliding stride of the convolution kernel.
15. The method according to claim 14 , further comprising:
performing a reverse operation on the n row vectors to obtain the M row vectors through restoration, wherein the reverse operation comprises one or more of sliding, splitting, or de-padding; and
restoring locations of the M row vectors in the S row vectors according to the read instruction for reading the M row vectors from the input dataset.
16. The method according to claim 11 , wherein the preprocessing comprises sliding.
17. The method according to claim 11 , wherein the preprocessing comprises padding.
18. The method according to claim 11 , wherein the preprocessing comprises concatenation.
19. The method according to claim 11 , wherein the preprocessing comprises sliding and concatenation.