Training generative adversarial networks
View Patent ↗Improving the training of generative adversarial networks by identifying the input and output layers in the generator network, assigning a gradient multiplier to each layer, and multiplying the gradient by the multiplier during back-propagation and before applying the product of the gradient and multiplier to node weights of the layers.
1. A computer implemented method for training machine learning systems, the method comprising:
identifying, by one or more computer processors, an input layer in a generator network;
assigning, by the one or more computer processors, a unique multiplier to a first network layer after the input layer, wherein the unique multiplier comprises a value associated with an inverse of a number of training set data instances;
applying, by the one or more computer processors, a gradient and the unique multiplier to a node weight of the first network layer during generator network back-propagation;
training, by the one or more computer processors, the generator network having the node weight;
providing, by the one or more computer processors, the trained generator network an input;
generating, by the trained generator network, an output; and
displaying the output to a user.
2. The computer implemented method according to claim 1 , further comprising assigning, by the one or more computer processors, a unique multiplier to each network layer.
3. The computer implemented method according to claim 2 , further comprising increasing, by the one or more computer processors, the unique multiplier according to a distance of a network layer from the input layer.
4. The computer implemented method according to claim 1 , further comprising assigning a unique multiplier to an output layer of the generator network, wherein the unique multiplier of the output layer of the generator network comprises a value of 1.0.
5. The computer implemented method according to claim 2 , wherein the unique multipliers of network layers comprise values interpolated between a unique multiplier for a network layer after the input layer to a unique multiplier for an output layer.
6. The computer implemented method according to claim 1 , further comprising:
identifying, by the one or more computer processors, a discriminator network input layer in a discriminator network;
assigning, by the one or more computer processors, a discriminator network multiplier to a discriminator network layer after the discriminator network input layer; and
applying, by the one or more computer processors, a discriminator network gradient and the discriminator network multiplier to a node weight of the discriminator network layer during discriminator network back-propagation.
7. A computer program product for training machine learning systems, the computer program product comprising one or more computer readable storage devices and program instructions collectively stored on the one or more computer readable storage devices, the stored program instructions comprising:
program instructions to identify an input layer in a generator network;
program instructions to assign a unique multiplier to a network layer after the input layer, wherein the unique multiplier comprises a value associated with an inverse of a number of training set data instances;
program instructions to apply a gradient and the unique multiplier to a node weight of the network layer during generator network back-propagation;
program instructions to train the generator network having the node weight;
program instructions to provide the trained generator network an input;
program instructions to generate an output; and
program instructions to display the output to a user.
8. The computer program product according to claim 7 , the stored program instructions further comprising program instructions to assign a unique multiplier to each network layer.
9. The computer program product according to claim 8 , the stored program instructions further comprising program instructions to increase the unique multiplier of each network layer according to a distance of the network layer from the input layer.
10. The computer program product according to claim 7 , further comprising assigning, by the one or more computer processors, a unique multiplier to an output layer of the generator network, wherein the unique multiplier of the output layer of the generator network comprises a value of 1.0.
11. The computer program product according to claim 7 , wherein the unique multipliers of network layers comprise values interpolated between a multiplier for a network layer after the input layer to the multiplier for an output layer.
12. The computer program product according to claim 7 , the stored program instructions further comprising:
program instructions to identify a discriminator network input layer in a discriminator network;
program instructions to assign a discriminator network multiplier to a discriminator network layer after the discriminator network input layer; and
program instructions to apply a discriminator network gradient and the discriminator network multiplier to a node weight of the discriminator network layer during discriminator network back-propagation.
13. A computer system for training machine learning systems, the computer system comprising:
one or more computer processors;
one or more computer readable storage devices; and
stored program instructions on the one or more computer readable storage devices for execution by the one or more computer processors, the stored program instructions comprising:
program instructions to identify an input layer in a generator network;
program instructions to assign a unique multiplier to a network layer after the input layer, wherein the unique multiplier comprises a value associated with an inverse of a number of training set data instances; and
program instructions to apply a gradient and the unique multiplier to a node weight of the network layer during back-propagation;
program instructions to train the generator network having the node weight;
program instructions to provide the trained generator network an input;
program instructions to generate an output; and
program instructions to display the output to a user.
14. The computer system according to claim 13 , the stored program instructions further comprising program instructions to assign a unique multiplier to each network layer.
15. The computer system according to claim 14 , the stored program instructions further comprising program instructions to increase the unique multiplier of each network layer according to a distance of the network layer from the input layer.
16. The computer system according to claim 13 , further comprising assigning, by the one or more computer processors, a unique multiplier to an output layer of the generator network, wherein the unique multiplier of the output layer of the generator network comprises a value of 1.0.
17. The computer system according to claim 13 , the stored program instructions further comprising:
program instructions to identify a discriminator network input layer in a discriminator network;
program instructions to assign a discriminator network multiplier to a discriminator network layer after the discriminator network input layer; and
program instructions to apply a discriminator network gradient and the discriminator network multiplier to a node weight of the discriminator network layer during discriminator network back-propagation.