METHODS AND SYSTEMS FOR OPTIMIZING A PEAK MEMORY USAGE OF AN ARTIFICIAL NEURAL NETWORK GRAPH

1 . A computer implemented method for optimizing memory usage of an artificial neural network graph comprising a plurality of layers and a plurality of tensors, the method-comprising the steps:

for each of the plurality of layers, determining a tensor working set, wherein the tensor working set comprises tensors that consume memory with respect to the respective layer;

determining whether at least one working set of the plurality of working sets requires memory usage above a pre-determined threshold;

if it is determined that at least one working set of the plurality of working sets requires memory usage above the pre-determined threshold, identifying a working set of the plurality of working sets which requires memory usage above the pre-determined threshold;

identifying at least one layer responsible for the memory usage above the pre-determined threshold in the identified working set; and

pruning the identified at least one layer.

2 . The computer implemented method of claim 1 ;

wherein the steps of identifying and pruning are repeated until every working set of the plurality of working sets requires memory below the pre-determined threshold.

3 . The computer implemented method of claim 2 ;

wherein in each step of identifying, the working set which requires a highest amount of memory, is identified.

4 . The computer implemented method according to claim 1 ;

wherein each layer comprises a respective plurality of channels; and

wherein pruning the at least one identified layer comprises reducing a number of channels of the at least one identified layer.

5 . The computer implemented method according to claim 1 ;

wherein pruning the at least one identified layer comprises removing the at least one identified layer.

6 . The computer implemented method according to claim 1 ;

wherein the working set of the plurality of working sets which requires maximum memory usage is determined based on an architecture of the artificial neural network graph.

7 . The computer implemented method according to claim 1 , further comprising:

determining an intermediate representation of the artificial neural network graph;

wherein the working set of the plurality of working sets which requires maximum memory usage is determined based on the intermediate representation.

8 . The computer implemented method according to claim 1 ;

wherein once every working set of the plurality of working sets requires memory below the pre-determined threshold, the artificial neural network graph after pruning is re-trained from scratch or fine-tuned from a previous training.

9 . The computer implemented method according to claim 1 ;

wherein the at least one identified layer is pruned based on an importance metric, wherein preferably the importance metric is provided by user input.

10 . The computer implemented method of claim 9 , wherein the importance metric is evaluated based on representative test data;

the computer implemented method preferably further comprising the following step:

training the artificial neural network graph before evaluating the importance metrics.

11 . The computer implemented method according to claim 1 , further comprising the following step:

generating a report comprising at least one of a layer summary report, a tensor summary report, or a working set summary report.

12 . The computer implemented method according to claim 1 , wherein the artificial neural network and or the pre-determined threshold are provided by user input.

13 . The computer implemented method according to claim 1 ;

wherein the artificial neural network graph is to be deployed on a resource-constrained embedded system after pruning;

wherein preferably the embedded system is a mobile computing device, a mobile phone, a tablet computing device, an automotive compute platform, or an edge device.

14 . A computer system comprising a plurality of computer hardware components configured to carry out steps of the computer implemented method according to claim 1 .

15 . A non-transitory computer readable medium comprising instructions for carrying out the computer implemented method according to claim 1 .