Apparatus for processing a deep learning model and a method thereof

1 . An apparatus for processing a deep learning model, the apparatus comprising:

a first memory;

a second memory;

a processor; and

a non-transitory storage medium having stored thereon computer-readable instructions that, when executed by the processor, cause the apparatus:

for each layer of the deep learning model, detect memory usage, a first processing time corresponding to the first memory being used, and a second processing time corresponding to the second memory being used, wherein the computer-readable instructions, when executed by the processor, cause the apparatus to i) in a process of determining a second node using a first node, detect memory usage corresponding to a first layer, a time that it takes to determine a second node when the first memory is used, and a time that it takes to determine the second node when the second memory is use and ii) in a process of determining a third node using the first node, detect memory usage corresponding to a second layer, a time that it takes to determine a third node when the first memory is used, and a time that it takes to determine the third node when the second memory is used,

determine an optimal memory for each layer of the deep learning model based on by optimizing an objective function based on the memory usage, the first processing time, and the second processing time,

allocate one or both of the first memory or the second memory to each of the first layer and the second layer according to the optimal memory for each of the first layer and the second layer, and

process the first layer and the second layer in parallel, wherein processing the first layer and the second layer includes i) processing the first layer using the one or both of the first memory or the second memory allocated to the first layer and i) in parallel with processing the first layer using the one or both of the first memory or the second memory allocated to the first layer, processing the second layer using the one or both of the first memory or the second memory allocated to the second layer.

2 . The apparatus of claim 1 , wherein:

the first memory includes a static random access memory (SRAM), and

the second memory includes a dynamic random access memory (DRAM).

3 . The apparatus of claim 1 , wherein the computer-readable instructions, when executed by the processor, cause the apparatus to release a memory allocated to the first layer and the second layer after the second node and the third node are determined.

4 . The apparatus of claim 3 , wherein the computer-readable instructions, when executed by the processor, cause the apparatus to, in a process of determining a fourth node based on the second node and the third node:

detect memory usage of a fourth layer corresponding to the second node, a time that it takes to process the fourth layer when the first memory is used, and a time that it takes to process the fourth layer when the second memory is used, and

detect memory usage of a fifth layer corresponding to the third node, a time that it takes to process the fifth layer when the first memory is used, and a time that it takes to process the fifth layer when the second memory is used.

5 . The apparatus of claim 4 , wherein the computer-readable instructions, when executed by the processor, cause the apparatus to release a memory allocated to the fourth layer and the fifth layer after the fourth node is determined.

6 . The apparatus of claim 5 , wherein the computer-readable instructions, when executed by the processor, cause the apparatus to, in a process of determining a fifth node based on the first node and the third node:

detect memory usage of a third layer corresponding to the first node, a time that it takes to process the third layer when the first memory is used, and a time that it takes to process the third layer when the second memory is used, and

detect memory usage of a sixth layer corresponding to the third node, a time that it takes to process the sixth layer when the first memory is used, and a time that it takes to process the sixth layer when the second memory is used.

7 . The apparatus of claim 6 , wherein the computer-readable instructions, when executed by the processor, cause the apparatus to release a memory allocated to the third layer and the sixth layer after the fifth node is determined.

8 . The apparatus of claim 7 , wherein the computer-readable instructions, when executed by the processor, cause the apparatus to, in a process of determining a sixth node based on the fourth node and the fifth node:

detect memory usage of a seventh layer corresponding to the fourth node, a time that it takes to process the seventh layer when the first memory is used, and a time that it takes to process the seventh layer when the second memory is used, and

detect memory usage of an eighth layer corresponding to the fifth node, a time that it takes to process the eighth layer when the first memory is used, and a time that it takes to process the eighth layer when the second memory is used.

9 . The apparatus of claim 8 , wherein the computer-readable instructions, when executed by the processor, cause the apparatus to release a memory allocated to the seventh layer and the eighth layer after the sixth node is determined.

10 . A method of processing a deep learning model, the method comprising:

for each layer of the deep learning model, detecting, by a processor, memory usage, a first processing time corresponding to a first memory being used, and a second processing time corresponding to a second memory being used, including i) in a process of determining a second node using a first node, detecting memory usage corresponding to a first layer, a time that it takes to determine a second node when the first memory is used, and a time that it takes to determine the second node when the second memory is used and ii) in a process of determining a third node using the first node, detecting memory usage corresponding to a second layer, a time that it takes to determine a third node when the first memory is used, and a time that it takes to determine the third node when the second memory is used;

determining, by the processor, an optimal memory for each layer of the deep learning model based on an objective function based on the memory usage, the first processing time, and the second processing time;

allocating, by the processor, one or both of the first memory or the second memory to each of the first layer and the second layer according to the optimal memory for each of the first layer and the second layer; and

processing, by the processor, the first layer and the second layer in parallel, including i) processing the first layer using the one or both of the first memory or the second memory allocated to the first layer and i) in parallel with processing the first layer using the one or both of the first memory or the second memory allocated to the first layer, processing the second layer using the one or both of the first memory or the second memory allocated to the second layer.

11 . The method of claim 10 , wherein:

the first memory includes a static random access memory (SRAM), and

the second memory includes a dynamic random access memory (DRAM).

12 . The method of claim 10 , further comprising releasing, by the processor, a memory allocated to the first layer and the second layer after the second node and the third node are determined.

13 . The method of claim 12 , wherein detecting the memory usage, the first processing time and the second processing time further includes, in a process of determining a fourth node based on the second node and a third node:

detecting memory usage of a fourth layer corresponding to the second node, a time that it takes to process the fourth layer when the first memory is used, and a time that it takes to process the fourth layer when the second memory is used, and

detecting memory usage of a fifth layer corresponding to the third node, a time that it takes to process the fifth layer when the first memory is used, and a time that it takes to process the fifth layer when the second memory is used.

14 . The method of claim 13 , further comprising releasing, by the processor, a memory allocated to the fourth layer and the fifth layer after the fourth node is determined.

15 . The method of claim 14 , wherein detecting the memory usage, the first processing time and the second processing time further includes, in a process of determining a fifth node based on the first node and the third node:

detecting memory usage of a third layer corresponding to the first node, a time that it takes to process the third layer when the first memory is used, and a time that it takes to process the third layer when the second memory is used, and

detecting memory usage of a sixth layer corresponding to the third node, a time that it takes to process the sixth layer when the first memory is used, and a time that it takes to process the sixth layer when the second memory is used.

16 . The method of claim 15 , further comprising releasing, by the processor, a memory allocated to the third layer and the sixth layer after the fifth node is determined.

17 . The method of claim 16 , wherein the detecting of the memory usage, the first processing time and the second processing time further includes, in a process of determining a sixth node based on the fourth node and the fifth node:

detecting memory usage of a seventh layer corresponding to the fourth node, a time that it takes to process the seventh layer when the first memory is used, and a time that it takes to process the seventh layer when the second memory is used, and

detecting memory usage of an eighth layer corresponding to the fifth node, a time that it takes to process the eighth layer when the first memory is used, and a time that it takes to process the eighth layer when the second memory is used.

18 . The method of claim 17 , further comprising releasing, by the processor, a memory allocated to the seventh layer and the eighth layer after the sixth node is determined.