Calculator, deep learning method and computer-readable recording medium storing program for deep learning
A deep learning method is performed by a computer having a plurality of nodes. The method comprises measuring, in learning performed by using the plurality of nodes in deep learning, performance of each of the nodes, allocating a number of batches to be processed by an entirety of the plurality of nodes to the individual nodes in accordance with the respective performance measured, and processing the allocated batches in each of the nodes.
1. A non-transitory computer-readable recording medium storing a program for causing a computer to execute a process, the process comprising:
measuring, in learning performed by using a plurality of nodes, each node of the plurality of nodes having a processing chip in deep learning, performance of each of the nodes;
after allocating a number of batches to be processed by an entirety of the plurality of nodes to each node of the plurality of nodes in accordance with a ratio of the respective performance measured, allocating by decreasing, by a predetermined size, the batches set for one of the nodes the performance of which is lowest and increasing, by the predetermined size, the batches set for one of the nodes the performance of which is highest; and
processing the allocated batches in each node of the plurality of nodes.
2. The recording medium according to claim 1 , wherein
the measuring of the performance and the allocating are performed every time for a predetermined number of times of repetitions related to the learning is reached.
3. The recording medium according to claim 1 , wherein
the measuring of the performance and the allocating are performed at predetermined time intervals.
4. A calculator comprising:
a memory, and
a processor coupled to the memory and configured to:
measure, in learning performed by using a plurality of nodes, each node of the plurality of nodes having a processing chip in deep learning, performance of each of the nodes;
after allocating a number of batches to be processed by an entirety of the plurality of nodes to each node of the plurality of nodes in accordance with a ratio of the respective performance measured, allocate by decreasing, by a predetermined size, the batches set for one of the nodes the performance of which is lowest and increasing, by the predetermined size, the batches set for one of the nodes the performance of which is highest; and
process the allocated batches in each node of the plurality of nodes.
5. The calculator according to claim 4 , wherein
the measure of the performance and the allocating in accordance with ratio of the respective performance measured are performed every time a predetermined number of times of repetitions related to the learning is reached.
6. The calculator according to claim 4 , wherein
the measure of the performance and the allocating are performed at predetermined time intervals.
7. A deep learning method performed by a computer, the method comprising:
measuring, in learning performed by using a plurality of nodes, each node of the plurality of nodes having a processing chip in deep learning, performance of each of the nodes;
after allocating a number of batches to be processed by an entirety of the plurality of nodes to each node of the plurality of nodes in accordance with the respective performance measured, allocating by decreasing, by a predetermined size, the batches set for one of the nodes the performance of which is lowest and increasing, by the predetermined size, the batches set for one of the nodes the performance of which is highest; and
processing the allocated batches in each node of the plurality of nodes.
8. The deep learning method according to claim 7 , wherein
the measuring of the performance and the allocating in accordance with ratio of the respective performance measured are performed every time a predetermined number of times of repetitions related to the learning is reached.
9. The deep learning method according to claim 7 , wherein
the measuring of the performance and the allocating in accordance with ratio of the respective performance measured are performed at predetermined time intervals.