Convolution computation engine, artificial intelligence chip, and data processing method
View Patent ↗This application discloses a convolution computation engine of a neural network model. The neural network model includes at least first and second network layers. The convolution computation engine includes a weight storage unit implemented by a first processing circuit and a data computation unit implemented by a second processing circuit. The weight storage unit pre-loads weight data of the second network layer from an external storage unit located outside the convolution computation engine during a convolution operation of the first network layer. The data computation unit obtains target input data of the second network layer in response to an operation instruction of the second network layer, and reads the weight data of the second network layer from the weight storage unit; and performs a convolution operation on the target input data by using the weight data of the second network layer.
1 . A convolution computation engine including a processor and a non-transitory memory, the convolution computation engine comprising:
a weight storage unit implemented on the processor and a data computation unit implemented on the processor, wherein:
an output end of the weight storage unit is connected to an input end of the data computation unit,
the weight storage unit is configured to pre-load weight data of a second network layer of a neural network model from an external storage unit during a convolution operation of a first network layer of the neural network model,
the data computation unit is configured to:
obtain target input data of the second network layer in response to an operation instruction for the second network layer,
read the weight data of the second network layer from the weight storage unit, and
perform a convolution operation on the target input data by using the weight data of the second network layer,
the target input data is split into a plurality of slices of data according to a size of a convolution kernel corresponding to a weight value in the weight data,
a size of each piece of data is less than or equal to the size of the convolution kernel corresponding to the weight value,
the data computation unit comprises H computation arrays, H being a positive integer,
some or all computation arrays of the H computation arrays perform convolution calculation on the plurality of slices of data in parallel, one computation array corresponding to one slice of data,
an h th computation array is configured to obtain a target slice of data of the target input data of the second network layer in response to the operation instruction for the second network layer, read the weight data of the second network layer from the weight storage unit, and perform the convolution operation on the target slice of data by using the weight data of the second network layer, h∈[1, H], the target slice of data being any slice of data in the target input data on which the convolution operation is not performed.
2 . The convolution computation engine according to claim 1 , wherein:
the weight data of the second network layer comprises weight values of N convolution kernels, N being a positive integer,
the weight storage unit comprises a decoder module and M storage arrays, M being a positive integer,
an output end of the decoder module is respectively connected to input ends of the storage arrays, and
the decoder module is configured to perform weight value read and write operations on the M storage arrays, each of the storage arrays being an array obtained by arranging a plurality of storage units in a form of a matrix.
3 . The convolution computation engine according to claim 2 , wherein the decoder module is configured to:
obtain the weight data of the second network layer from the external storage unit during the convolution operation of the first network layer,
determine storage addresses of the weight values in the weight data of the second network layer in the M storage arrays, and
respectively write the weight values into the M storage arrays in sequence according to the storage addresses of the weight values.
4 . The convolution computation engine according to claim 2 , wherein the decoder module is configured to:
obtain storage addresses of the weight values in the weight data of the second network layer in the M storage arrays in response to a weight read instruction of the second network layer,
respectively read the weight values from the M storage arrays according to the storage addresses of the weight values, and
transmit the read weight values into the data computation unit.
5 . The convolution computation engine according to claim 4 , wherein:
the decoder module comprises: a first decoder, a second decoder, and an access controller,
an input end of the second decoder is connected to an output end of the first decoder,
an output end of the second decoder is connected to the access controller,
the first decoder is configured to decode array identifiers of the storage arrays,
the second decoder is configured to decode storage coordinates of the storage units in the storage arrays, and
the access controller is configured to generate access timing of the M storage arrays.
6 . The convolution computation engine according to claim 5 , wherein the access timing comprises at least one of the following: initialization timing, read and write control and switch timing, write timing, read timing, or error processing timing.
7 . The convolution computation engine according to claim 5 , wherein:
the second decoder comprises: a column decoder and a row decoder,
the column decoder is configured to decode column coordinates in the storage coordinates of the storage units in the storage arrays, and
the row decoder is configured to decode row coordinates in the storage coordinates of the storage units in the storage arrays.
8 . The convolution computation engine according to claim 5 , wherein a storage address of a target weight value in an n th convolution kernel in the weight data in the M storage arrays comprises an array identifier of a target storage array configured to store the target weight value and storage coordinates of a target storage unit configured to store the target weight value in the target storage array, n∈[1, N], and the target weight value being any weight value in the n th convolution kernel.
9 . The convolution computation engine according to claim 8 , wherein:
the first decoder is configured to select the target storage array configured to store the target weight value from the M storage arrays and determine the array identifier of the target storage array,
the second decoder is configured to select the target storage unit configured to store the target weight value from the target storage array and determine the storage coordinates of the target storage unit, and
the access controller is configured to obtain the target weight value from the external storage unit during the convolution operation of the first network layer, and write the target weight value into the target storage unit in the target storage array according to the array identifier corresponding to the target weight value and the storage coordinates corresponding to the target weight value.
10 . The convolution computation engine according to claim 5 , wherein a storage address of a target weight value in an n th convolution kernel in the weight data in the M storage arrays comprises an array identifier of a target storage array configured to store the target weight value and storage coordinates of a target storage unit configured to store the target weight value in the target storage array, n∈[1, N], and the target weight value being any weight value in the n th convolution kernel.
11 . The convolution computation engine according to claim 10 , wherein:
the first decoder is configured to decode the storage address corresponding to the target weight value in response to the weight read instruction of the second network layer, to obtain the array identifier of the target storage array configured to store the target weight value,
the second decoder is configured to decode the storage address corresponding to the target weight value, to obtain the storage coordinates of the target storage unit configured to store the target weight value in the target storage array, and
the access controller is configured to determine the target storage array from the M storage arrays according to the array identifier corresponding to the target weight value, read the target weight value from the target storage unit in the target storage array according to the storage coordinates corresponding to the target weight value, and transmit the read target weight value into the data computation unit.
12 . The convolution computation engine according to claim 8 , wherein after reading the target weight value from the target storage array, the access controller is further configured to: perform error processing on the read target weight value when the read target weight value is abnormal,
the error processing meaning that data correction is performed on the read target weight value by using a parity bit corresponding to the target weight value, and the parity bit corresponding to the target weight value being generated when the target weight value is written into the target storage array.
13 . The convolution computation engine according to claim 10 , wherein:
the h th computation array is an array of P rows and Q columns formed by P*Q multiply accumulate modules, both P and Q being positive integers,
each of the multiply accumulate modules comprises: a first input end, a second input end, an upper-level input end, and a module output end,
a module output end of a multiply accumulate module in a p th row and a q th column is connected to an upper-level input end of a multiply accumulate module in a (p+1) th row and the q th column, p∈[1, P−1] and q∈[1, Q],
the target slice of data is divided into a plurality of pieces of two-dimensional data according to a depth of the target slice of data, and any two-dimensional data is inputted into first input ends of multiply accumulate modules in any row in the h th computation array in a broadcast manner,
the weight data of the second network layer comprises weight values of N convolution kernels, N being a positive integer and N being less than or equal to Q, and
a weight value of an n th convolution kernel is inputted into second input ends of multiply accumulate modules in an n th column in the h th computation array.
14 . The convolution computation engine according to claim 13 , wherein each of the multiply accumulate modules further comprises: a mode selection end configured to select a fixed-point arithmetic mode or a floating-point arithmetic mode, a fixed-point general-purpose unit, a floating-point special-purpose unit, and an output selection unit.
15 . The convolution computation engine according to claim 14 , wherein:
the fixed-point general-purpose unit is respectively connected to the first input end, the second input end, the upper-level input end, and the mode selection end of each of the multiply accumulate modules,
a fixed-point output end of the fixed-point general-purpose unit is respectively connected to the output selection unit and the floating-point special-purpose unit,
the floating-point special-purpose unit is respectively connected to the first input end, the second input end, the upper-level input end, the fixed-point output end, and the mode selection end of each of the multiply accumulate modules,
a floating-point output end of the floating-point special-purpose unit is connected to the output selection unit, and
the output selection unit is configured to connect the fixed-point output end to the module output end when an arithmetic mode indicated by the mode selection end is the fixed-point arithmetic mode, and connect the floating-point output end to the module output end when the arithmetic mode is the floating-point arithmetic mode.
16 . The convolution computation engine according to claim 1 , wherein the convolution computation engine is integrated into one of a central processing unit, a field programmable gate array, an application-specific integrated circuit, a graphics processing unit, or an artificial intelligence chip.
17 . The convolution computation engine according to claim 1 , the weight storage unit being implemented by a first processing circuit, the data computation unit being implemented by a second processing circuit, and the first processing circuit and the second processing circuit being independent of each other.
18 . A data processing method performed by the convolution computation engine according to claim 1 , and the data processing method comprising:
pre-loading weight data of the second network layer during a convolution operation of the first network layer,
reading target input data of the second network layer in response to an operation instruction for the second network layer, and
performing a convolution operation on the target input data by using the weight data of the second network layer.
19 . An artificial intelligence chip, comprising the convolution computation engine of the neural network model according to claim 1 .