SELECTING HYPER PARAMETERS FOR MACHINE LEARNING ALGORITHMS BASED ON PAST TRAINING RESULTS

1 . A method for selecting hyper parameters for machine learning algorithms based on past training results, the method comprising:

obtaining a first group of values of hyper parameters, a second group of values of hyper parameters and a third group of values of hyper parameters, the second group of values of hyper parameters differs from the first group of values of hyper parameters, and the third group of values of hyper parameters differs from the first group of values of hyper parameters and the second group of values of hyper parameters;

obtaining a first result of training a machine learning algorithm using a first plurality of training examples and the first group of values of hyper parameters;

obtaining a second result of training the machine learning algorithm using a second plurality of training examples and the second group of values of hyper parameters;

obtaining a third result of training the machine learning algorithm using a third plurality of training examples and the third group of values of hyper parameters; and

using the first result, the second result, the third result, the first group of values of hyper parameters, the second group of values of hyper parameters and the third group of values of hyper parameters to select at least one value of a hyper parameter for a prospective training of the machine learning algorithm.

2 . The method of claim 1 , wherein the first group of values of hyper parameters comprises a first learning rate value, the second group of values of hyper parameters comprises a second learning rate value, the third group of values of hyper parameters comprises a third learning rate value, and further comprising using the first result, the second result, the third result, the first learning rate value, the second learning rate value and the third learning rate value to select a learning rate value for the prospective training of the machine learning algorithm.

3 . The method of claim 1 , wherein the first group of values of hyper parameters comprises a first selection of a learning rate update method, the second group of values of hyper parameters comprises a second selection of a learning rate update method, the third group of values of hyper parameters comprises a third selection of a learning rate update method, and further comprising using the first result, the second result, the third result, the first selection of a learning rate update method, the second selection of a learning rate update method and the third selection of a learning rate update method to select a learning rate update method for the prospective training of the machine learning algorithm.

4 . The method of claim 1 , wherein the first group of values of hyper parameters comprises a first batch size, the second group of values of hyper parameters comprises a second batch size, the third group of values of hyper parameters comprises a third batch size, and further comprising using the first result, the second result, the third result, the first batch size, the second batch size and the third batch size to select a batch size for the prospective training of the machine learning algorithm.

5 . The method of claim 1 , wherein the first group of values of hyper parameters comprises a first network structure, the second group of values of hyper parameters comprises a second network structure, the third group of values of hyper parameters comprises a third network structure, and further comprising using the first result, the second result, the third result, the first network structure, the second network structure and the third network structure to select a network structure for the prospective training of the machine learning algorithm.

6 . The method of claim 1 , wherein the first group of values of hyper parameters comprises a first number of layers, the second group of values of hyper parameters comprises a second number of layers, the third group of values of hyper parameters comprises a third number of layers, and further comprising using the first result, the second result, the third result, the first number of layers, the second number of layers and the third number of layers to select a number of layers for the prospective training of the machine learning algorithm.

7 . The method of claim 1 , wherein the first group of values of hyper parameters comprises a first number of artificial neurons, the second group of values of hyper parameters comprises a second number of artificial neurons, the third group of values of hyper parameters comprises a third number of artificial neurons, and further comprising using the first result, the second result, the third result, the first number of artificial neurons, the second number of artificial neurons and the third number of artificial neurons to select a number of artificial neurons for the prospective training of the machine learning algorithm.

8 . The method of claim 1 , wherein the first group of values of hyper parameters comprises a first selection of an activation function, the second group of values of hyper parameters comprises a second selection of an activation function, the third group of values of hyper parameters comprises a third selection of an activation function, and further comprising using the first result, the second result, the third result, the first selection of an activation function, the second selection of an activation function and the third selection of an activation function to select an activation function for the prospective training of the machine learning algorithm.

9 . The method of claim 1 , wherein the first group of values of hyper parameters comprises a first selection of a loss function, the second group of values of hyper parameters comprises a second selection of a loss function, the third group of values of hyper parameters comprises a third selection of a loss function, and further comprising using the first result, the second result, the third result, the first selection of a loss function, the second selection of a loss function and the third selection of a loss function to select a loss function for the prospective training of the machine learning algorithm.

10 . The method of claim 1 , wherein the first group of values of hyper parameters comprises a first selection of an initialization method, the second group of values of hyper parameters comprises a second selection of an initialization method, the third group of values of hyper parameters comprises a third selection of an initialization method, and further comprising using the first result, the second result, the third result, the first selection of an initialization method, the second selection of an initialization method and the third selection of an initialization method to select an initialization method for the prospective training of the machine learning algorithm.

11 . The method of claim 1 , wherein the first group of values of hyper parameters comprises a first selection of a kernel function, the second group of values of hyper parameters comprises a second selection of a kernel function, the third group of values of hyper parameters comprises a third selection of a kernel function, and further comprising using the first result, the second result, the third result, the first selection of a kernel function, the second selection of a kernel function and the third selection of a kernel function to select a kernel function for the prospective training of the machine learning algorithm.

12 . The method of claim 1 , wherein the first group of values of hyper parameters comprises a first selection of a distance measure, the second group of values of hyper parameters comprises a second selection of a distance measure, the third group of values of hyper parameters comprises a third selection of a distance measure, and further comprising using the first result, the second result, the third result, the first selection of a distance measure, the second selection of a distance measure and the third selection of a distance measure to select a distance measure for the prospective training of the machine learning algorithm.

13 . The method of claim 1 , wherein the first group of values of hyper parameters comprises a first selection of a stopping condition, the second group of values of hyper parameters comprises a second selection of a stopping condition, the third group of values of hyper parameters comprises a third selection of a stopping condition, and further comprising using the first result, the second result, the third result, the first selection of a stopping condition, the second selection of a stopping condition and the third selection of a stopping condition to select a stopping condition for the prospective training of the machine learning algorithm.

14 . The method of claim 1 , wherein the first group of values of hyper parameters comprises a first sampling criterion used to sample the first plurality of training examples from a base group of training examples, the second group of values of hyper parameters comprises a second sampling criterion used to sample the second plurality of training examples from the base group of training examples, the third group of values of hyper parameters comprises a third sampling criterion used to sample the third plurality of training examples from a base group of training examples, and further comprising using the first result, the second result, the third result, the first sampling criterion, the second sampling criterion and the third sampling criterion to select a sampling criterion for sampling a plurality of training examples from the base group of training examples for the prospective training of the machine learning algorithm.

15 . The method of claim 1 , wherein further comprising:

training the machine learning algorithm using the first plurality of training examples and the first group of values of hyper parameters to generate a first inference model;

using the first inference model and a plurality of testing examples to generate a first plurality of outputs;

comparing the first plurality of outputs and a plurality of desired results to determine the first result.

16 . The method of claim 1 , wherein further comprising:

training the machine learning algorithm using the first plurality of training examples and the first group of values of hyper parameters to generate a first inference model;

using the first inference model and a plurality of testing examples to generate a first plurality of outputs;

providing information based on the first plurality of outputs to at least one user;

receiving one or more feedbacks related to the information based on the first plurality of outputs from the at least one user; and

using the one or more feedbacks related to the information based on the first plurality of outputs to determine the first result.

17 . The method of claim 1 , wherein further comprising:

determining an embedding in a mathematical space of at least a value of the first group of values of hyper parameters, a value of the second group of values of hyper parameters and a value of the third group of values of hyper parameters; and

using the determined embedding of the values in the mathematical space to select the at least one value of the hyper parameter for the prospective training of the machine learning algorithm.

18 . The method of claim 17 , further comprising:

determining whether the first result is preferable to the second result and the third result;

in response to the determination that the first result is preferable to the second result and the third result, selecting the at least one value of the hyper parameter for the prospective training of the machine learning algorithm to be closer to the first group of values of hyper parameters than to the second group of values of hyper parameters and to the third group of values of hyper parameters according to the embedding in the mathematical space; and

in response to the determination that the first result is not preferable to the second result and the third result, selecting the at least one value of the hyper parameter for the prospective training of the machine learning algorithm to be farther from the first group of values of hyper parameters than from at least one of the second group of values of hyper parameters and the third group of values of hyper parameters according to the embedding in the mathematical space.

19 . A system for selecting hyper parameters for machine learning algorithms based on past training results, the system comprising:

at least one processor configured to:

obtain a first group of values of hyper parameters, a second group of values of hyper parameters and a third group of values of hyper parameters, the second group of values of hyper parameters differs from the first group of values of hyper parameters, and the third group of values of hyper parameters differs from the first group of values of hyper parameters and the second group of values of hyper parameters;

obtain a first result of training a machine learning algorithm using a first plurality of training examples and the first group of values of hyper parameters;

obtain a second result of training the machine learning algorithm using a second plurality of training examples and the second group of values of hyper parameters;

obtain a third result of training the machine learning algorithm using a third plurality of training examples and the third group of values of hyper parameters; and

use the first result, the second result, the third result, the first group of values of hyper parameters, the second group of values of hyper parameters and the third group of values of hyper parameters to select at least one value of a hyper parameter for a prospective training of the machine learning algorithm.

20 . A non-transitory computer readable medium storing data and computer implementable instructions for carrying out a method for selecting hyper parameters for machine learning algorithms based on past training results, the method comprising:

obtaining a first group of values of hyper parameters, a second group of values of hyper parameters and a third group of values of hyper parameters, the second group of values of hyper parameters differs from the first group of values of hyper parameters, and the third group of values of hyper parameters differs from the first group of values of hyper parameters and the second group of values of hyper parameters;

obtaining a first result of training a machine learning algorithm using a first plurality of training examples and the first group of values of hyper parameters;

obtaining a second result of training the machine learning algorithm using a second plurality of training examples and the second group of values of hyper parameters;

obtaining a third result of training the machine learning algorithm using a third plurality of training examples and the third group of values of hyper parameters; and

using the first result, the second result, the third result, the first group of values of hyper parameters, the second group of values of hyper parameters and the third group of values of hyper parameters to select at least one value of a hyper parameter for a prospective training of the machine learning algorithm.