Real-time resource usage reduction in artificial neural networks
A generated algorithm used by a neural network is captured during execution of an iteration of the neural network. A candidate algorithm is identified based on the generated algorithm. A determination is made that the candidate algorithm utilizes less memory than the generated algorithm. Based on the determination the neural network is updated by replacing the generated algorithm with the candidate algorithm.
1. A method comprising:
capturing, during execution of an iteration of a neural network, a generated algorithm used by the neural network during the iteration, wherein the execution of the iteration is a forward propagation and a backward propagation of the neural network;
identifying, based on the generated algorithm, a candidate algorithm;
recording, during execution of the iteration, the memory utilization of the generated algorithm;
performing, outside of the neural network, a forward propagation and a backward propagation with the candidate algorithm;
comparing the memory usage of the recorded generated algorithm and the performed candidate algorithm;
determining that the candidate algorithm utilizes less memory than the generated algorithm; and
updating, based on the determination, the neural network by replacing the generated algorithm of the neural network with the candidate algorithm.
2. The method of claim 1 , further comprising:
detecting, before the determining, that the neural network has finished execution of the iteration;
pausing execution of the neural network; and
resuming, after the updating, the neural network.
3. The method of claim 1 , wherein the neural network is a define-by-run neural network and wherein the identifying the generated algorithm is an identifying an updated layout of the neural network.
4. The method of claim 1 , wherein the neural network is a convolutional neural network.
5. The method of claim 4 , wherein the generated algorithm is a convolutional algorithm.
6. The method of claim 1 , wherein the method further comprises:
monitoring usage memory allocated to the neural network after the replacement of the generated algorithm with the candidate algorithm; and
flagging, based on the monitoring the memory usage, a subset of the memory allocated to the neural network.
7. The method of claim 6 , wherein the neural network comprises a second algorithm, wherein the method further comprises:
assigning the flagged subset of the memory allocated to the neural network to the second algorithm of the neural network.
8. The method of claim 6 , wherein the method further comprises:
deallocating the flagged subset of memory.
9. A system comprising:
a memory; and
a processor, the processor communicatively coupled to the memory, the processor configured to perform a method comprising:
capturing, during execution of an iteration of a neural network, a generated algorithm used by the neural network during the iteration, wherein the execution of the iteration is a forward propagation and a backward propagation of the neural network;
identifying, based on the generated algorithm, a candidate algorithm;
recording, during execution of the iteration, the memory utilization of the generated algorithm;
performing, outside of the neural network, a forward propagation and a backward propagation with the candidate algorithm;
comparing the memory usage of the recorded generated algorithm and the performed candidate algorithm;
determining that the candidate algorithm utilizes less memory than the generated algorithm; and
updating, based on the determination, the neural network by replacing the generated algorithm of the neural network with the candidate algorithm.
10. The system of claim 9 , wherein the method further comprises:
detecting, before the determining, that the neural network has finished execution of the iteration;
pausing execution of the neural network; and
resuming, after the updating, the neural network.
11. The system of claim 9 , wherein the neural network is a define-by-run neural network and wherein the identifying the generated algorithm is an identifying an updated layout of the neural network.
12. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to perform a method comprising:
capturing, during execution of an iteration of a neural network, a generated algorithm used by the neural network during the iteration, wherein the execution of the iteration is a forward propagation and a backward propagation of the neural network;
identifying, based on the generated algorithm, a candidate algorithm;
recording, during execution of the iteration, the memory utilization of the generated algorithm;
performing, outside of the neural network, a forward propagation and a backward propagation with the candidate algorithm;
comparing the memory usage of the recorded generated algorithm and the performed candidate algorithm;
determining that the candidate algorithm utilizes less memory than the generated algorithm; and
updating, based on the determination, the neural network by replacing the generated algorithm of the neural network with the candidate algorithm.
13. The computer program product of claim 12 , wherein the method further comprises:
detecting, before the determining, that the neural network has finished execution of the iteration;
pausing execution of the neural network; and
resuming, after the updating, the neural network.
14. The computer program product of claim 12 , wherein the neural network is a convolutional neural network.
15. The computer program product of claim 14 , wherein the generated algorithm is a convolutional algorithm.
16. The computer program product of claim 12 , wherein the method further comprises:
monitoring usage memory allocated to the neural network after the replacement of the generated algorithm with the candidate algorithm; and
flagging, based on the monitoring the memory usage, a subset of the memory allocated to the neural network.