Method and apparatus for performing different types of convolution operations with the same processing elements
A method for implementing a convolutional neural network (CNN) accelerator on a target includes utilizing one or more processing elements to perform convolution. A configuration of the CNN accelerator is modified to change filters implemented by the CNN accelerator and to change formatting of output data. The one or more processing elements are utilized to perform one of deconvolution and backpropagation convolution in response to the change in the filters and formatting of the output data.
1. A method for implementing a convolutional neural network (CNN) accelerator on a target, comprising:
determining whether a deconvolution operation is to be performed;
extracting a plurality of convolution filters from a deconvolution filter associated with the deconvolution operation;
utilizing one or more processing elements to perform convolution on deconvolution input data using the plurality of convolution filters to generate convolution results;
interlacing the convolution results to produce deconvolution output data;
reformatting a convolution filter to a backpropagation convolution filter;
generating an additional plurality of convolution filters from the backpropagation convolution filter; and
interlacing additional convolution results, generated from performing convolution on backpropagation convolution input data using the additional plurality of convolution filters, to produce backpropagation convolution output data.
2. The method of claim 1 further comprising:
utilizing the one or more processing elements to perform deconvolution.
3. The method of claim 1 , wherein extracting the plurality of convolution filters from the deconvolution filter comprises:
extracting smaller convolution filters from the deconvolution filter based on stride; and
reflecting each of the smaller convolution filters along x and y axes.
4. The method of claim 1 , wherein reformatting the convolution filter comprises repacking the convolution filter by transforming a convolution layer having k filters of depth c to c filters of depth k.
5. The method of claim 1 , wherein extracting the plurality of convolution filters from the deconvolution filter is performed during runtime of the CNN accelerator by transformation elements residing on the CNN accelerator.
6. The method of claim 1 , wherein extracting the plurality of convolution filters from the deconvolution filter is performed prior to runtime of the CNN accelerator.
7. The method of claim 1 , wherein the one or more processing elements performs convolution, deconvolution, and backward propagation convolution at different instances of time.
8. The method of claim 1 , wherein each of the processing elements comprises:
a cache;
a computation unit that performs dot product operations; and
an accumulation unit.
9. A convolutional neural network (CNN) accelerator implemented on a target, comprising:
a sequencer unit that coordinates a first data flow between components on the target during a first configuration and that coordinates a second data flow between the components on the target during a second configuration;
transformation elements comprising a convolution filter extraction unit that generates a plurality of convolution filters from a deconvolution filter; and
a plurality of processing elements that performs convolution on deconvolution input data using the plurality of convolution filters to generate convolution results during the first configuration, and that performs one of deconvolution and backpropagation convolution during the second configuration,
wherein the transformation elements further comprise an interlacing unit that interlaces the convolution results to produce deconvolution output data.
10. The CNN accelerator of claim 9 further comprising:
a CNN algorithm identification unit that identifies a CNN algorithm to be executed by the CNN accelerator.
11. The CNN accelerator of claim 9 , wherein the transformation elements further comprise:
a filter repacking unit that reformats a convolution filter to a backpropagation convolution filter,
wherein the convolution filter extraction unit generates an additional plurality of convolution filters from the backpropagation convolution filter; and
wherein the interlacing unit interlaces additional convolution results, generated from performing convolution on backpropagation convolution input data using the additional plurality of convolution filters, to produce backpropagation convolution output data.
12. The CNN accelerator of claim 9 , wherein the convolution filter extraction unit generates the plurality of convolution filters from the deconvolution filter during runtime of the CNN accelerator.
13. The CNN accelerator of claim 11 , wherein the convolution filter extraction unit generates the additional plurality of convolution filters from the backpropagation convolution filter during runtime of the CNN accelerator.
14. The CNN accelerator of claim 9 , wherein the one or more processing elements performs convolution, deconvolution, and backward propagation convolution at different instances of time.
15. The CNN accelerator of claim 9 , wherein each of the processing elements comprises:
a cache;
a computation unit that performs dot product operations; and
an accumulation unit.
16. A non-transitory computer readable medium including a sequence of instructions stored thereon for causing a computer to execute a method for implementing a convolutional neural network (CNN) accelerator on a target, comprising:
reformatting a convolution filter to a backpropagation convolution filter;
generating a plurality of convolution filters from the backpropagation convolution filter;
utilizing one or more processing elements to perform convolution on backpropagation convolution input data using the plurality of convolution filters to generate convolution results; and
interlacing the convolution results to produce backpropagation convolution output data.
17. The non-transitory computer readable medium of claim 16 further comprising:
utilizing the one or more processing elements to perform backpropagation convolution.
18. The non-transitory computer readable medium of claim 16 , wherein the one or more processing elements performs convolution, deconvolution, and backward propagation convolution at different instances of time.
19. The non-transitory computer readable medium of claim 16 further comprising:
generating an additional plurality of convolution filters from a deconvolution filter; and
interlacing additional convolution results, generated from performing convolution on deconvolution input data using the additional plurality of convolution filters, to produce deconvolution output data.