Method for managing a convolutional computation and corresponding device

1 . A method for managing a convolutional calculation carried out by a calculation circuit adapted to calculate output data on output channels from convolution kernels applied to input data blocks on at least one input channel, wherein calculations on each input data block correspond respectively to an output datum on an output channel, and wherein the calculations with each convolution kernel correspond to the output data on each output channel respectively, the method comprising:

identifying a size of a memory location available in a temporary working memory of the calculation circuit;

determining whether the temporary working memory has sufficient capacity for storing a desired number of convolutional kernels for a current convolutional layer;

allocating a memory location of a buffer memory of the calculation circuit having a size of a minimum size threshold in response to determining that the temporary working memory lacks sufficient capacity and that the size of the memory location available in the temporary working memory is smaller than the minimum size threshold;

pre-loading a maximum number of the convolution kernels storable in the allocated memory location of the buffer memory, wherein the pre-loading occurs after the allocating step and uses the allocated buffer memory, and wherein the maximum number is determined by the available space within the allocated memory location; and

controlling the calculation circuit to calculate a set of output data calculable from the convolution kernels pre-loaded in the allocated buffer memory.

2 . The method according to claim 1 , wherein controlling the calculation circuit to calculate the set of output data comprises successively loading, in the calculation circuit, the input data blocks corresponding to the set of output data.

3 . The method according to claim 1 , further comprising repeatedly pre-loading in the temporary working memory the maximum number of the convolution kernels and repeatedly controlling the calculation circuit to calculate the set of the output data until the set of output data in all output channels is calculated.

4 . The method according to claim 1 , wherein the set of output data corresponds to a maximum size of the output data receivable at one time in each output channel, and wherein the set comprises an entire output data of each output channel or only a segment of the output data of each output channel.

5 . The method according to claim 4 , wherein the segment of the output data of each output channel corresponds to a row or a packet of rows of output data.

6 . The method according to claim 4 , wherein, in response to the set only comprising the segment of the output data of each output channel:

repeatedly preloading convolution kernels that are different from those of the previous pre-loading and repeatedly controlling for the same segment, until all the output data of the segment is calculated for all output channels; and

renewing preloading for other segments, until all the output data in all other segments of all the output channels is calculated.

7 . The method according to claim 1 , wherein the convolution kernels comprise weight data, and wherein pre-loading comprises reorganizing the weight data of the convolution kernels to optimize the calculation of the output data.

8 . The method according to claim 1 , wherein the convolution kernels comprise weight data, and wherein controlling the calculation circuit to calculate the set of output data comprises multiplying and accumulating the input data block loaded with the weight data of the pre-loaded convolution kernels.

9 . The method according to claim 1 , further comprising, before pre-loading the convolution kernels in the temporary working memory, storing the convolution kernels in a non-volatile memory internal or external to the calculation circuit, and, while controlling the calculation circuit, storing the input data and the output data in the volatile memory internal or external to the calculation circuit.

10 . A device comprising:

a calculation circuit configured to calculate output data on output channels from convolution kernels applied to input data blocks on at least one input channel so that calculations on each input data block correspond respectively to each output datum on the output channels, and so that calculations with each convolution kernel correspond to the output data on each output channel respectively; and

a processor configured to manage a convolutional calculation performed by the calculation circuit,

wherein the processor is further configured for:

identifying a size of a memory location available in a temporary working memory of the calculation circuit;

determining whether the temporary working memory has sufficient capacity for storing a desired number of convolutional kernels for a current convolutional layer;

allocating a memory location of a buffer memory of the calculation circuit having a size of a minimum size threshold in response to determining that the temporary working memory lacks sufficient capacity and that the size of the memory location available in the temporary working memory is smaller than the minimum size threshold;

pre-loading a maximum number of the convolution kernels storable in the allocated memory location of the buffer memory, wherein pre-loading occurs after allocating and uses the allocated buffer memory, and wherein the maximum number is determined by the available space within the allocated memory location; and

controlling the calculation circuit to calculate a set of output data calculable from the convolution kernels pre-loaded in the allocated buffer memory.

11 . The device according to claim 10 , wherein the processor is configured for controlling the calculation circuit to calculate the set of output data from the pre-loaded convolution kernels by successively loading the input data blocks corresponding to the set of output data in the calculation circuit.

12 . The device according to claim 10 , wherein the processor is configured for repeatedly pre-loading in the temporary working memory the maximum number of the convolution kernels and repeatedly controlling the calculation circuit to calculate the set of output data until the set of output data in all output channels is calculated.

13 . The device according to claim 10 , wherein the set of output data corresponds to a maximum size of the output data receivable at one time in each output channel, and wherein the set of output data comprises an entire output data of each output channel, or only a segment of the output data of each output channel.

14 . The device according to claim 13 , wherein the segment of the output data of each output channel corresponds to a row or to a packet of rows of each output channel.

15 . The device according to claim 13 , wherein the processor is configured for:

repeating pre-loading with the convolution kernels different from those of the previous pre-loading and repeating the controlling for the same segment until all the output data of the segment is calculated for all output channels in response to the set of output data only comprising the segment; and

renewing preloading for other segments, until all the output data in all other segments of all the output channels is calculated.

16 . The device according to claim 10 , wherein the convolution kernels comprise weight data, and wherein the processor is configured for, during pre-loading, reorganizing the weight data of the convolution kernels and for optimizing the calculation of the output data by the calculation circuit.

17 . The device according to claim 10 , wherein the convolution kernels comprise weight data, and wherein the calculation circuit is configured for calculating the output data by multiplying and accumulating the input data of the blocks loaded with the weight data of the pre-loaded convolution kernels.

18 . A non-transitory computer readable storage medium comprising instructions for managing a convolutional calculation carried out by a calculation circuit adapted to calculate output data on output channels from convolution kernels applied to input data blocks on at least one input channel, wherein calculations on each input data block correspond respectively to an output datum on an output channel, and wherein the calculations with each convolution kernel correspond to the output data on each output channel respectively, the instructions, when executed by a processor, cause the processor to:

identify a size of a memory location available in a temporary working memory of the calculation circuit;

determine whether the temporary working memory has sufficient capacity for storing a desired number of convolutional kernels for a current convolutional layer;

allocate a memory location of a buffer memory of the calculation circuit having a size of a minimum size threshold in response to determining that the temporary working memory lacks sufficient capacity and that the size of the memory location available in the temporary working memory is smaller than the minimum size threshold;

pre-load a maximum number of the convolution kernels storable in the allocated memory location of the buffer memory, wherein pre-loading occurs after the allocating step and uses the allocated buffer memory, and wherein the maximum number is determined by the available space within the allocated memory location; and

control the calculation circuit to calculate a set of output data calculable from the convolution kernels pre-loaded in the allocated buffer memory.

19 . The non-transitory computer readable medium of claim 18 , wherein controlling the calculation circuit to calculate the set of output data comprises successively loading, in the calculation circuit, the input data blocks corresponding to the set of output data.

20 . The non-transitory computer readable medium of claim 18 , wherein the instructions, when executed by a processor, cause the processor to repeatedly pre-loading in the temporary working memory the maximum number of the convolution kernels and repeatedly controlling the calculation circuit to calculate the set of the output data until the set of output data in all output channels is calculated.