Apparatus for performing convolution operations in a convolutional neural network

1. An apparatus included in a computer, the apparatus for performing a convolution operation in a sparsified convolutional neural network, comprising:

a first selector coupled to a first memory of the computer storing weight parameters, and configured to select one or more first nonzero elements of one or more kernels of the weight parameters in one or more first channels, each first channel including at least one first nonzero element, and each first nonzero element being located at a same element position in the corresponding kernel, the element position being a two-dimensional location in two dimensions of height and width;

a second selector coupled to a second memory of the computer storing input feature data, and configured to select one or more first data items of the input feature data in the one or more first channels corresponding to the one or more first nonzero elements, a data item position of each first data item in the input feature data being dependent on the element position of the corresponding first nonzero element, and each first data item being located in the same first channel with the corresponding first nonzero element, the data item position being a two-dimensional location in the two dimensions of height and width; and

a calculator including a multiplication accumulator array configured to multiply each first data item from the second selector with the corresponding first nonzero element from the first selector to obtain a product, and accumulate the product.

2. The apparatus of claim 1 , further comprising a first counter, the first selector being configured to select one or more second nonzero elements of the one or more kernels in one or more second channels according to a current counting value from the first counter.

3. The apparatus of claim 2 , wherein the first selector is configured to select the one or more second nonzero elements of the one or more kernels in the one or more second channels by selecting all nonzero elements in each of the one or more kernels among all kernels of the weight parameters, the one or more kernels corresponding to the current counting value of the first counter, the one or more second channels including the one or more first channels, and the one or more second nonzero elements including the one or more first nonzero elements.

4. The apparatus of claim 2 , wherein the first selector is configured to select the one or more second nonzero elements of the one or more kernels in the one or more second channels by selecting one or more nonzero elements in one or more channels of each of the one or more kernels at an element position corresponding to the current counting value of the first counter, the one or more second channels including the one or more first channels, and the one or more second nonzero elements including the one or more first nonzero elements.

5. The apparatus of claim 2 , wherein the first selector is configured to select the one or more second nonzero elements of the one or more kernels in the one or more second channels by selecting one or more nonzero elements of the one or more kernels in one or more second channels corresponding to the current counting value of the first counter, the one or more second channels including the one or more first channels, and the one or more second nonzero elements including the one or more first nonzero elements.

6. The apparatus of claim 2 , further comprising a second counter, the first selector being further configured to select the one or more first nonzero elements from the one or more second nonzero elements, the one or more first channels corresponding to a current counting value of the second counter.

7. The apparatus of claim 2 , wherein the first selector includes one or more multiplexers.

8. The apparatus of claim 7 , wherein the first selector further comprises:

a first buffer memory configured to buffer the one or more second nonzero elements.

9. The apparatus of claim 1 , further comprising a first counter, the second selector being configured to select one or more second data items of the input feature data in one or more third channels according to a current counting value from the first counter.

10. The apparatus of claim 9 , wherein the one or more second data items of the input feature data in the one or more third channels correspond to one or more data items in output feature data corresponding to the current counting value of the first counter, the one or more third channels including the one or more first channels, and the one or more second data items including the one or more first data items.

11. The apparatus of claim 9 , wherein the second selector is configured to select the one or more second data items of the input feature data in the one or more third channels by:

retrieving the element position from a nonzero element position list by using the current counting value of the first counter as an index value of the nonzero element position list, the nonzero element position list recording element positions of nonzero elements in the one or more kernels;

determining one or more data item positions in the input feature data corresponding to the element position; and

selecting the one or more second data items in the input feature data at each of the determined data item positions in the one or more third channels, the one or more third channels including the one or more first channels, and the one or more second data items including the one or more first data items.

12. The apparatus of claim 9 , wherein the one or more third channels correspond to the current count value of the first counter, the one or more third channels including the one or more first channels, and the one or more second data items including the one or more first data items.

13. The apparatus of claim 9 , further comprising a second counter, and the second selector being further configured to

retrieve the one or more first channels from a nonzero channel list by using a current counting value of the second counter as an index value of the nonzero channel list, the nonzero channel list recording channels in which nonzero elements in the one or more kernels locate; and

select the one or more first data items in the one or more first channels from the one or more second data items.

14. The apparatus of claim 13 , wherein the information recorded in the nonzero channel list also relates to the element positions of the nonzero elements in the one or more kernels.

15. The apparatus of claim 9 , wherein the second selector includes one or more multiplexers.

16. The apparatus of claim 15 , wherein the second selector further comprises:

a second buffer memory configured to buffer the one or more second data items.

17. The apparatus of claim 1 , wherein the calculator is further configured to perform at least one of activation and quantization operations.

18. A computer-implemented method for performing a convolution operation in a sparsified convolutional neural network, comprising:

selecting, by a first selector coupled to a first memory of a computer storing weight parameters, one or more first nonzero elements of one or more kernels of the weight parameters in one or more first channels, each first channel including at least one first nonzero element, and each first nonzero element being located at a same element position in the corresponding kernel, the element position being a two-dimensional location in two dimensions of height and width;

selecting, by a second selector coupled to a second memory of the computer storing input feature data, one or more first data items of the input feature data in the one or more first channels corresponding to the one or more first nonzero elements, a data item position of each first data item in the input feature data being dependent on the element position of the corresponding first nonzero element, and each first data item being located in the same first channel with the corresponding first nonzero element, the data item position being a two-dimensional location in the two dimensions of height and width; and

multiplying, by a multiplication accumulator array of a calculator, each first data item from the second selector with the corresponding first nonzero element from the first selector to obtain a product, and accumulating the product.

19. The method of claim 18 , wherein selecting one or more first nonzero elements of one or more kernels of the weight parameters in one or more first channels comprises:

selecting one or more second nonzero elements of the one or more kernels in one or more second channels according to a counting value from a first counter; and

selecting the one or more first nonzero elements from the one or more second nonzero elements, the one or more first channels corresponding to a counting value of a second counter.

20. The method of claim 18 , wherein selecting one or more first data items of the input feature data in the one or more first channels comprises:

selecting one or more second data items of the input feature data in one or more third channels according to a counting value of the first counter;

retrieving the one or more first channels from a nonzero channel list by using the counting value of the second counter as an index value of the nonzero channel list, the nonzero channel list recording channels in which nonzero elements in the one or more kernels locate; and

selecting the one or more first data items in the one or more first channels from the one or more second data items.