Winograd convolution operation method, apparatus, and device, and storage medium
The present disclosure provides a winograd convolution operation method, a winograd convolution operation apparatus, a device, and a storage medium. The apparatus includes: processors and a memory, where the memory is configured to store a program code, and the processors are configured to call the program code stored in the memory and execute the operation method. Through the operation method, a system, the device and the storage medium of the present disclosure, performance loss of a computer system may be reduced, and operation speed may be improved. Through the present disclosure, processing efficiency may be improved.
1 . A winograd convolution operation method, comprising:
splitting, by a splitting circuit, data in a winograd convolution operation into a plurality of sub-tensors during a transformation process of the winograd convolution operation;
obtaining, by a transformation and summation operation circuit, meta-tensors corresponding to a plurality of sub-tensors;
executing, by the transformation and summation operation circuit, winograd transformations of the meta-tensors;
transforming, by a transformation and summation operation circuit, the plurality of sub-tensors to obtain winograd transformation results of the plurality of sub-tensors based on the winograd transformation results of the meta-tensors;
summing, by the transformation and summation operation circuit, the winograd transformation results of the plurality of sub-tensors to obtain a winograd transformation result of the data, wherein
a summation operation of the winograd transformation results of the plurality of sub-tensors is completed, by a plurality of operation sub-apparatuses, based on set strategies that includes distributing, by the transformation and summation operation circuit, the summation operation of the winograd transformation results of the plurality of sub-tensors to the plurality of operation sub-apparatuses for an execution based on a principle of balancing loads of the plurality of operation sub-apparatuses, and
the distributing of the summation operation of the winograd transformation results of the plurality of sub-tensors to the plurality of operation sub-apparatuses for the execution based on the principle of balancing the loads of the plurality of operation sub-apparatuses includes:
determining a number of non-zero elements in each transformation matrix corresponding to each sub-tensor of the plurality of sub-tensors;
determining an order of the winograd transformation results of the plurality of sub-tensors in the summation operation according to the number of the non-zero elements; and
completing, by a convolution operation circuit, the winograd convolution operation according to the winograd transformation result of the data.
2 . The winograd convolution operation method of claim 1 , wherein
the distributing of the summation operation of the winograd transformation results of the plurality of sub-tensors to the plurality of operation sub-apparatuses for the execution based on the principle of balancing the loads of the plurality of operation sub-apparatuses further includes:
distributing addition execution tasks to the plurality of operation sub-apparatuses correspondingly according to a determined order of the winograd transformation results of the plurality of sub-tensors in the summation operation and a preset mapping relationship between element positions in a result matrix and the plurality of operation sub-apparatuses, wherein
the addition execution tasks are used to calculate each element in the result matrix,
additive numbers in the addition execution tasks of each element in the result matrix are composed of non-zero element values in the data that is formed by the winograd transformation results of the plurality of sub-tensors, and
the summation operation at a same element position in the result matrix is performed by a same operation sub-apparatus.
3 . The winograd convolution operation method of claim 2 , wherein before the distributing of the summation operation of the winograd transformation results of the plurality of sub-tensors to the plurality of operation sub-apparatuses for the execution based on the principle of balancing the loads of the plurality of operation sub-apparatuses, the method further comprises:
determining a number of addition executions required for the summation operation according to the winograd transformation results of the plurality of sub-tensors;
determining a number of clocks according to a size of the plurality of sub-tensors or the result matrix; and
determining a number of the operation sub-apparatuses according to the number of the clocks and the number of the addition executions required for the summation operation.
4 . The winograd convolution operation method of claim 3 , wherein the distributing of the summation operation of the winograd transformation results of the plurality of sub-tensors to the plurality of operation sub-apparatuses for the execution based on the principle of balancing the loads of the plurality of operation sub-apparatuses includes:
distributing the addition execution tasks to the plurality of operation sub-apparatuses correspondingly according to the determined order of the winograd transformation results of the plurality of sub-tensors in the summation operation and the preset mapping relationship between the element positions in the result matrix and the operation sub-apparatuses, so as to complete addition calculations of each element in the result matrix within the number of the clocks.
5 . A winograd convolution operation apparatus, comprising:
a splitting circuit configured to split data in a winograd convolution operation into a plurality of sub-tensors during a transformation process of the winograd convolution operation;
a transformation and summation operation circuit configured to:
obtain meta-tensors corresponding to a plurality of sub-tensors;
execute winograd transformations of the meta-tensors;
transform the plurality of sub-tensors to obtain winograd transformation results of the plurality of sub-tensors and sum the winograd transformation results of the plurality of sub-tensors to obtain a winograd transformation result of the data, wherein
a summation operation of the winograd transformation results of the plurality of sub-tensors is completed, by a plurality of operation sub-apparatuses, based on set strategies, that includes during the summation operation of the winograd transformation results of the plurality of sub-tensors, wherein
the transformation and summation operation circuit is further configured to distribute the summation operation of the winograd transformation results of the plurality of sub-tensors to the plurality of operation sub-apparatuses for an execution based on a principle of balancing loads of the plurality of operation sub-apparatuses;
when the summation operation of the winograd transformation results of the plurality of sub-tensors is distributed to the plurality of operation sub-apparatuses based on the principle of balancing the loads of the plurality of operation sub-apparatuses, the transformation and summation operation circuit is further configured to:
determine a number of non-zero elements in each transformation matrix corresponding to each sub-tensor of the plurality of sub-tensors;
determine an order of the winograd transformation results of the plurality of sub-tensors in the summation operation according to the number of the non-zero elements; and
a convolution operation circuit configured to complete the winograd convolution operation according to the winograd transformation result of the data.
6 . The winograd convolution operation apparatus of claim 5 , wherein the winograd convolution operation apparatus is applied to a master processor or a slave processor of a master-slave structure.
7 . The winograd convolution operation apparatus of claim 5 , wherein
when the summation operation of the winograd transformation results of the plurality of sub-tensors is distributed to the plurality of operation sub-apparatuses based on the principle of balancing the loads of the plurality of operation sub-apparatuses, the transformation and summation operation circuit is further configured to:
distribute addition execution tasks to the plurality of operation sub-apparatuses correspondingly according to a determined order of the winograd transformation results of the plurality of sub-tensors in the summation operation and a preset mapping relationship between element positions in a result matrix and operation sub-apparatuses, wherein
the addition execution tasks are used to calculate each element in the result matrix,
additive numbers in the addition execution tasks of each element in the result matrix are composed of non-zero element values in the data that is formed by the winograd transformation results of the plurality of sub-tensors, and
the summation operation at a same element position in the result matrix is performed by a same operation sub-apparatus.
8 . The winograd convolution operation apparatus of claim 7 , further comprising:
an addition-execution-number determination circuit configured to determine a number of addition executions required for the summation operation according to the winograd transformation results of the plurality of sub-tensors;
a clock number determination circuit configured to determine a number of clocks according to a size of the plurality of sub-tensors or the result matrix; and
an operation-sub-apparatus-number determination circuit configured to determine a number of the operation sub-apparatuses according to the number of the clocks and the number of the addition executions required for the summation operation.
9 . The winograd convolution operation apparatus of claim 8 , wherein the clock number determination circuit is specifically configured to:
determine a maximum size in the sub-tensors and the result matrix; and
determine the number of the clocks according to the maximum size.
10 . The winograd convolution operation apparatus of claim 8 , wherein the operation-sub-apparatus-number determination circuit is specifically configured to:
calculate a ratio of the number of the addition executions required to calculate a summation result to the number of the clocks used to calculate the summation result; and
determine the number of the operation sub-apparatuses according to the ratio, wherein the number of the operation sub-apparatuses is greater than the ratio.
11 . The winograd convolution operation apparatus of claim 8 , wherein the addition-execution-number determination circuit is specifically configured to:
determine a non-zero element value at the same element position in data that is formed by a winograd transformation result of each sub-tensor as an additive number of the element position in the result matrix;
determine the number of summation operations at each element position according to the additive number of each element position in the result matrix; and
determine a sum of the number of summation operations at each element position as the number of the addition executions required to calculate a summation result.
12 . The winograd convolution operation apparatus of claim 5 , wherein when the addition execution tasks used to calculate each element in the result matrix are distributed to the plurality of the operation sub-apparatuses correspondingly to complete the addition calculations of each element in the result matrix within a preset number of the clocks, the transformation and summation operation circuit is specifically configured to:
input, within first N consecutive clocks, one of additive numbers used to calculate each element position in the result matrix to a corresponding operation sub-apparatus to perform an initial assignment on each element in the result matrix; and
input, within remaining clocks, remaining additive numbers used to calculate each element position in the result matrix to corresponding operation sub-apparatuses to perform the addition calculations until each element value in the result matrix is obtained, wherein all operation sub-apparatuses are in a processing state within each clock.
13 . The winograd convolution operation apparatus of claim 5 , wherein when the addition execution tasks used to calculate each element in the result matrix are distributed to the plurality of operation sub-apparatuses correspondingly to complete the addition calculations of each element in the result matrix within a preset number of the clocks, the transformation and summation operation circuit is specifically configured to:
divide the number of the clocks into a plurality of clock segments and divide each element position in the result matrix into a plurality of element position sets, wherein the clock segments correspond to the element position sets one to one;
for each element position set, within a first clock of a corresponding clock segment, input one of additive numbers used to calculate all element positions in the element position set to a corresponding operation sub-apparatus to perform an initial assignment on each element position in the element position set; and
within remaining clocks of the clock segment, input remaining additive numbers used to calculate all element positions in each element position set to corresponding operation sub-apparatuses to perform the addition calculations until each element value in the element position set is obtained, wherein all operation sub-apparatuses are in a processing state within each clock.
14 . The winograd convolution operation apparatus of claim 5 , wherein
the splitting circuit is further configured to parse the data to obtain the plurality of sub-tensors,
the data is a sum of the plurality of sub-tensors,
a number of the plurality of sub-tensors is the same as a number of non-zero elements in the data,
each of the sub-tensors of the plurality of sub-tensors has a single non-zero element, and
the non-zero elements in the sub-tensors of the plurality of sub-tensors are the same as the non-zero elements in corresponding positions in the data.
15 . The winograd convolution operation apparatus of claim 5 , wherein when the plurality of sub-tensors is transformed to obtain the winograd transformation results of the plurality of sub-tensors, the transformation and summation operation circuit is further configured to:
obtain a winograd transformation result of a meta-tensor corresponding to each sub-tensor of the plurality of sub-tensors, wherein the meta-tensor is a tensor that sets a non-zero element of a sub-tensor as 1;
set a non-zero element value of the sub-tensor of the plurality of sub-tensors as a coefficient to be multiplied by the winograd transformation result of the meta-tensor corresponding to the sub-tensor of the plurality of sub-tensors to obtain a winograd transformation result of the sub-tensor; and
sum the winograd transformation results of the plurality of sub-tensors to obtain the winograd transformation result of the data.
16 . The winograd convolution operation apparatus of claim 15 , wherein
when the winograd transformation result of the meta-tensor corresponding to each sub-tensor is obtained, the transformation and summation operation circuit is further configured to:
for each sub-tensor of the plurality of sub-tensors, multiply a left side of the meta-tensor corresponding to the sub-tensor of the plurality of sub-tensors by a left multiplication matrix and multiply a right side of the meta-tensor corresponding to the sub-tensor of the plurality of sub-tensors by a right multiplication matrix, so as to obtain the winograd transformation result of the meta-tensor,
both the left multiplication matrix and the right multiplication matrix are determined by a size of the sub-tensor and a type of winograd transformation, and
the type of winograd transformation includes a winograd forward transformation and a winograd inverse transformation.