Method and non-transitory computer readable medium for compute-in-memory macro arrangement, and electronic device applying the same
A method and a non-transitory computer readable medium for CIM arrangement, and an electronic device applying the same are proposed. The method for CIM arrangement includes to obtain information of the number of CIM macros and information of the dimension of each of the CIM micros, to obtain information of the number of input channels and the number of output channels of a designated convolutional layer of a designate neural network, and to determine a CIM macro arrangement for arranging the CIM macros according to the number of the CIM macros, the dimension of each of the CIM macros, the number of the input channels and the number of the output channels of the designated convolutional layer of the designated neural network, for applying convolution operation to the input channels to generate the output channels.
1 . A method for compute-in-memory (CIM) macro arrangement comprising:
obtaining information of the number of a plurality of CIM macros and information of a dimension of each of the CIM macros;
obtaining information of the number of a plurality of input channels and the number of a plurality of output channels of a designated convolutional layer of a designated neural network; and
determining a CIM macro arrangement for arranging the CIM macros according to the number of the CIM macros, the dimension of each of the CIM macros, the number of the input channels and the number of the output channels of the designated convolutional layer of the designated neural network, for applying convolution operation to the input channels to generate the output channels,
wherein the determined CIM macro arrangement provides a summation of a vertical dimension of the CIM macros adapted for performing convolution of a plurality of filters and the input channels of the designated convolution layer by a minimum number of times for batch loading the input channels.
2 . The method according to claim 1 , wherein the step of determining the CIM macro arrangement according to the number of the CIM macros, the dimensions of each of the CIM macros, and the number of the input channels and the number of the output channels of the designated convolutional layer of the designated neural network comprises:
determining the CIM macro arrangement capable of performing the convolution of the filters and the input channels according to latency, energy consumption, and utilization.
3 . The method according to claim 2 ,
wherein the latency is associated with at least one of a DRAM latency, a latency for loading weights into the CIM macros, and a processing time of the CIM macros,
wherein the energy consumption is associated with energy cost for accessing at least one memory including an on-chip SRAM which is in a same chip as the CIM macros and a DRAM outside the chip, and
wherein the utilization is a ratio of used part of the CIM macros to all of the CIM macros.
4 . The method according to claim 1 , wherein the determined CIM macro arrangement further provides a summation of a horizontal dimension of the CIM macros adapted for performing the convolution of the filters and the input channels of the designated convolution layer by a minimum number of times for batch loading the filters.
5 . An electronic apparatus comprising:
a plurality of compute-in-memory (CIM) macros, wherein the CIM macros are arranged in a predetermined CIM macro arrangement based on the number of the CIM macros, the dimensions of each of the CIM macros, and the number of a plurality of input channels and the number of a plurality of output channels of a designated convolutional layer of a designated neural network, wherein the predetermined CIM macro arrangement provides a summation of a vertical dimension of the CIM macros adapted for performing convolution of a plurality of filters and the input channels of the designated convolution layer by a minimum number of times for batch loading the input channels; and
a processing circuit, configured to:
load weights in the arranged CIM macros; and
input a plurality of input channels of one input feature map into the arranged CIM macros with the loaded weights for a convolutional operation to generate an output activation of one of a plurality of output feature maps.
6 . The electronic apparatus according to claim 5 ,
wherein the processing circuit loads the weights of the filters in the arranged CIM macros based on the predetermined CIM macro arrangement, the number of the filters, height and width of each kernel of a plurality of kernels of each of the filters and the number of the kernels in each filter, wherein each of the kernels of each filter is respectively applied to a corresponding one of the input channels of the designated convolutional layer of the designated neural network.
7 . The electronic apparatus according to claim 5 ,
wherein the processing circuit loads each of the filters into the arranged CIM macros columnwisely.
8 . The electronic apparatus according to claim 5 ,
wherein the processing circuit determines whether to batch loads the weights of the plurality of filters in the arranged CIM macros based on the height and width of each kernel and a summation of a horizontal dimension of the arranged CIM macro.
9 . A non-transitory computer readable medium storing a program causing a computer to:
obtaining information of the number of a plurality of CIM macros and information of a dimension of each of the CIM macros;
obtaining information of the number of a plurality of input channels and the number of a plurality of output channels of a designated convolutional layer of a designated neural network; and
determining a CIM macro arrangement for arranging the CIM macros according to the number of the CIM macros, the dimension of each of the CIM macros, the number of the input channels and the number of the output channels of the designated convolutional layer of the designated neural network, for applying convolution operation to the input channels to generate the output channels,
wherein the determined CIM macro arrangement provides a summation of a vertical dimension of the CIM macros adapted for performing convolution of a plurality of filters and the input channels of the designated convolution layer by a minimum number of times for batch loading the input channels.
10 . A method for compute-in-memory (CIM) macro arrangement comprising:
obtaining information of the number of a plurality of CIM macros and information of a dimension of each of the CIM macros;
obtaining information of the number of a plurality of input channels and the number of a plurality of output channels of a designated convolutional layer of a designated neural network; and
determining a CIM macro arrangement for arranging the CIM macros according to the number of the CIM macros, the dimension of each of the CIM macros, the number of the input channels and the number of the output channels of the designated convolutional layer of the designated neural network, for applying convolution operation to the input channels to generate the output channels,
wherein the determined CIM macro arrangement provides a summation of a horizontal dimension of the CIM macros adapted for performing convolution of a plurality of filters and the input channels of the designated convolution layer by a minimum number of times for batch loading the filters.
11 . An electronic apparatus comprising:
a plurality of compute-in-memory (CIM) macros, wherein the CIM macros are arranged in a predetermined CIM macro arrangement based on the number of the CIM macros, the dimensions of each of the CIM macros, and the number of a plurality of input channels and the number of a plurality of output channels of a designated convolutional layer of a designated neural network, wherein the determined CIM macro arrangement provides a summation of a horizontal dimension of the CIM macros adapted for performing convolution of a plurality of filters and the input channels of the designated convolution layer by a minimum number of times for batch loading the filters; and
a processing circuit, configured to:
load weights in the arranged CIM macros; and
input a plurality of input channels of one input feature map into the arranged CIM macros with the loaded weights for a convolutional operation to generate an output activation of one of a plurality of output feature maps.
12 . A non-transitory computer readable medium storing a program causing a computer to:
obtaining information of the number of a plurality of CIM macros and information of a dimension of each of the CIM macros;
obtaining information of the number of a plurality of input channels and the number of a plurality of output channels of a designated convolutional layer of a designated neural network; and
determining a CIM macro arrangement for arranging the CIM macros according to the number of the CIM macros, the dimension of each of the CIM macros, the number of the input channels and the number of the output channels of the designated convolutional layer of the designated neural network, for applying convolution operation to the input channels to generate the output channels,
wherein the determined CIM macro arrangement provides a summation of a horizontal dimension of the CIM macros adapted for performing convolution of a plurality of filters and the input channels of the designated convolution layer by a minimum number of times for batch loading the filters.