Accurate ensemble by mutating neural network parameters
View Patent ↗A computer-implemented method is provided for reducing training costs for an ensemble of machine-learning-based classifiers. The method includes training, by a processor, a given machine-learning-based classifier from among the ensemble to obtain a trained classifier. The method further includes dumping, by the processor, parameters used to train the trained classifier to obtain dumped parameters. The method also includes recording, by the processor, change rates of each of the dumped parameters. The method additionally includes creating, by the processor without training, a new classifier from at least one other machine-learning-based classifier in the ensemble by calculating the dumped parameters plus change rates times random numbers for each local prediction by the trained classifier.
1 . A computer-implemented method for reducing training costs for an ensemble of machine-learning-based classifiers, the method comprising:
training, by a processor, a given machine-learning-based classifier from among the ensemble to obtain a trained classifier defined by non-hyper parameters;
dumping, by the processor, the non-hyper parameters of the trained classifier to obtain dumped parameters;
recording, by the processor, change rates of each of the dumped parameters; and
mutating, by the processor, the trained classifier to generate a new classifier in the ensemble by, for each local prediction by the trained classifier, combining a random number and the change rates to with the dumped parameters to generate new non-hyper parameters that define the new classifier, wherein the combining generates the new classifier by while avoiding training and its requisite costs to improve computer efficiency.
2 . The computer-implemented method of claim 1 , wherein the dumped parameters used to train the trained classifier are connection weights in the given machine-learning-based classifier.
3 . The computer-implemented method of claim 1 , wherein said recording step is performed responsive to said dumping step.
4 . The computer-implemented method of claim 1 , wherein the new classifier is created for inference use in an absence of explicitly training the new classifier.
5 . The computer-implemented method of claim 1 , wherein the mutating mutates the dumped parameters based on the recorded change rate to avoid prediction accuracy degradation by the new classifier.
6 . The computer-implemented method of claim 1 , wherein the random numbers are taken from a limited range of random numbers.
7 . The computer-implemented method of claim 1 , wherein the change rates are from a start time to an end time of a final training epoch.
8 . The computer-implement method of claim 1 , wherein the change rates are from an intermediate training epoch to a final training epoch, and wherein the intermediate training epoch immediately precedes the final training epoch in a sequence of training epochs including the intermediate training epoch and the final training epoch.
9 . The computer-implemented method of claim 1 , wherein multiple random numbers are used to create the new classifier, each of the multiple random numbers corresponding to a respective different one of a plurality of machine-learning-based classifier layers in the given machine-learning-based classifier.
10 . The computer implemented method of claim 1 , wherein multiple random numbers are used to create the new classifiers, each of the multiple random numbers corresponding to a different one of the dumped parameters.
11 . A computer program product for reducing training costs for an ensemble of machine-learning-based classifiers, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:
training, by a processor of the computer, a given machine-learning-based classifier from among the ensemble to obtain a trained classifier defined by non-hyper parameters;
dumping, by the processor, the non-hyper parameters of the trained classifier to obtain dumped parameters;
recording, by the processor, change rates of each of the dumped parameters; and
mutating, by the processor, the trained classifier to generate a new classifier in the ensemble by, for each local prediction by the trained classifier, combining a random number and the change rates with the dumped parameters to generate new non-hyper parameters that define the new classifier, wherein the combining generates the new classifier while avoiding training and its requisite costs to improve computer efficiency.
12 . The computer program product of claim 11 , wherein the dumped parameters used to train the trained classifier are connection weights in the given machine-learning-based classifier.
13 . The computer program product of claim 11 , wherein said recording step is performed responsive to said dumping step.
14 . The computer program product of claim 11 , wherein the new classifier is created for inference use in an absence of explicitly training the new classifier.
15 . The computer program product of claim 11 , wherein mutating mutates the dumped parameters based on the recorded change rate to avoid prediction accuracy degradation by the new classifier.
16 . The computer program product of claim 11 , wherein the random numbers are taken from a limited range of random numbers.
17 . The computer program product of claim 11 , wherein the change rates are from a start time to an end time of a final training epoch.
18 . The computer program product of claim 11 , wherein the change rates are from an intermediate training epoch to a final training epoch, and wherein the intermediate training epoch immediately precedes the final training epoch in a sequence of training epocha including the intermediate training epoch and the final training epoch.
19 . The computer program product of claim 11 , wherein multiple random numbers are used to create the new classifier, each of the multiple random numbers corresponding to a respective different one of a plurality of machine-learning-based classifier layers in the given machine-learning-based classifier.
20 . A computer processing system for reducing training costa for an ensemble of machine-learning-based classifiers, the system comprising:
a memory for storing program code; and
a hardware processor for running the program code to
train a given machine-learning-based classifier from among the ensemble to obtain a trained classifier defined by non-hyper parameters;
dump the non-hyper parameters of the trained classifier to obtain dumped parameters;
record change rates of each of the dumped parameters; and
mutate the trained classifier to generate a new trained classifier in the ensemble by, for each local prediction by the trained classifier, combining a random number and the change rates with the dumped parameters to generate new non-hyper parameters that define the new classifier, wherein the combining generates the new classifier while avoiding training and its requisite costs to improve computer efficiency.