Dividing neural networks
A method of implementing a neural network in a neuromorphic apparatus having a memory and processing circuitry, where the method includes dividing, by the processing circuitry, the neural network into a plurality of sub-networks based on a size of a core of the memory to implement the neural network, initializing, by the processing circuitry, a hyper-parameter used in the sub-networks, and training, by the processing circuitry, the sub-networks by using the hyper-parameter.
1. A method of implementing a neural network in a neuromorphic apparatus having a memory and processing circuitry, the method comprising:
dividing, by the processing circuitry, the neural network into a plurality of sub-networks by,
determining a number of axon circuits included in a core of the memory,
determining a number of input activations to the neural network, and
dividing the neural network into the plurality of sub-networks based on the number of axon circuits and the number of input activations such that a number of the input activations associated with each of the sub-networks is equal to the number of axon circuits;
initializing, by the processing circuitry, a hyper-parameter used in the sub-networks; and
training, by the processing circuitry, the sub-networks by using the hyper-parameter.
2. The method of claim 1 , wherein the initializing further comprises initializing, by the processing circuitry, the hyper-parameter with an initial value based on the number of axon circuits included in the core.
3. The method of claim 1 , wherein the initializing further comprises initializing, by the processing circuitry, the hyper-parameter with a value selected from among values indicating a standard deviation of input activations of a sub-network of the neural network.
4. The method of claim 3 , wherein the standard deviation is based on a number of the input activations of the sub-network with a number of sub-networks.
5. The method of claim 1 , wherein the dividing further comprises:
arranging the plurality of sub-networks in stages such that each stage includes a number of the sub-networks arranged in parallel that together have a number of the input activations equal to a number of output activations of a prior stage.
6. The method of claim 5 , further comprising:
mapping each of the plurality of sub-networks in a same stage to different ones of a plurality of cores.
7. The method of claim 1 , further comprising:
mapping, by the processing circuitry, the sub-networks to the memory, and
driving the sub-networks by the processing circuitry.
8. The method of claim 1 , wherein an element corresponding to a synapse in the core comprises a variable resistance element.
9. A non-transitory computer-readable recording medium having recorded thereon a program for executing the method of claim 1 on a computer.
10. An apparatus for implementing a neural network, the apparatus comprising:
a memory comprising a core; and
processing circuitry configured to drive the neural network by,
dividing the neural network into a plurality of sub-networks by,
determining a number of axon circuits included in the core of the memory,
determining a number of input activations to the neural network, and
dividing the neural network into the plurality of sub-networks based on the number of axon circuits and the number of input activations such that a number of the input activations associated with each of the sub-networks is equal to the number of axon circuits;
initializing a hyper-parameter used in the sub-networks; and
training the sub-networks by using the hyper-parameter.
11. The apparatus of claim 10 , wherein the processing circuitry is further configured to initialize the hyper-parameter with an initial value selected based on the number of axon circuits included in the core.
12. The apparatus of claim 10 , wherein the processing circuitry is further configured to initialize the hyper-parameter with a value selected from among values indicating a standard deviation.
13. The apparatus of claim 12 , wherein the processing circuitry is further configured to initialize the hyper-parameter by calculating the standard deviation by merging a number of input activations of the sub-network with a number of sub-networks.
14. The apparatus of claim 10 , wherein the processing circuitry is further configured to drive the neural network by generating at least one intermediate activation by dividing the neural network.
15. The apparatus of claim 10 , wherein the processing circuitry is further configured to drive the neural network by mapping the sub-networks to the memory and drive the sub-networks.
16. The apparatus of claim 10 , wherein an element corresponding to a synapse in the core comprises a variable resistance element.
17. The apparatus of claim 10 , wherein the processing circuitry is further configured to divide the neural network into the plurality of sub-networks by,
arranging the plurality of sub-networks in stages such that each stage includes a number of the sub-networks arranged in parallel that together have a number of the input activations equal to a number of output activations of a prior stage.
18. The apparatus of claim 17 , wherein the processing circuitry is further configured to map each of the plurality of sub-networks in a same stage to different ones of a plurality of cores.