Systems and methods for implementing an intelligent machine learning optimization platform for multiple tuning criteria

1. A system for tuning hyperparameters for improving an effectiveness including one or more objective performance metrics of a model, the system comprising:

a remote tuning service for tuning hyperparameters of a model of a subscriber to the remote tuning service, wherein the remote tuning service is hosted on a distributed network of computers that:

receives a multi-criteria tuning work request for tuning hyperparameters of the model of the subscriber to the remote tuning service, wherein the multi-criteria tuning work request includes at least:

(i) a first objective function of the model to be optimized by the remote tuning service;

(ii) a second objective function to be optimized by the remote tuning service, the second objective function being distinct from the first objective function; computes:

a first conditionally constrained joint function for the model based on subjecting the first objective function to the second objective function;

a second conditionally constrained joint function for the model based on subjecting the second objective function to the first objective function of the model;

executes a tuning operation of the hyperparameters for the model based on a tuning of the first conditionally constrained joint function and the second conditionally constrained joint function;

simultaneously optimizes the first objective function of the model and the second objective function of the model during the tuning operation; and

identifies a set of proposed hyperparameter values that simultaneously optimizes both of the first objective function of the model and the second objective function of the model based on a hyperparameter-based point along a non-convex Pareto optimal curve; and

switches a tuning mode of the remote tuning service from a first standard tuning mode to a second non-convex tuning mode for optimizing the first objective function and the second objective function of the multi-criteria tuning request based on identifying a best fit curve of the non-convex Pareto optimal curve.

2. The system according to claim 1 , wherein:

the first conditionally constrained joint function relates to a first singular objective function that enables the remote tuning service to optimize the first objective function of the model subject to constraints of the second objective function of the model, and

the second conditionally constrained joint function relates to a second singular objective function that enables the remote tuning service to optimize the second objective function of the model subject to constraints of the first objective function of the model.

3. The system according to claim 1 , wherein:

the first objective function is represented as ƒ1(x, y) and the second objective function is represented as ƒ2(x, y);

the first conditionally constrained joint function comprises:

Maximize ƒ1( x,y )

Subject to: ƒ2( x,y )>=ε1,ε2,ε3 , . . . εN,

x and y relate to potential hyperparameter values selectable from a multi-dimensional coordinate system; and

each ε1, ε2, ε3, . . . εN represents a distinct epsilon constraint derived from the second objective function ƒ2(x, y).

4. The system according to claim 1 , wherein the remote tuning service further:

derives a plurality of epsilon constraint values for the first conditionally constrained function based on the second objective function, wherein the deriving includes:

identifying a range of possible hyperparameter values for the second objective function based on a predetermined minimum value and a predetermined maximum value of the second objective function;

identifying a plurality of distinct and non-overlapping segments of hyperparameter values for the second objective function based on the range;

associating each of the plurality of distinct and non-overlapping segments values to a distinct one of the plurality of epsilon constraint values.

5. The system according to claim 1 , wherein the remote tuning service further:

derives a plurality of epsilon constraint values for the second conditionally constrained function based on the first objective function, wherein the deriving includes:

identifying a range of possible hyperparameter values for the first objective function based on a predetermined minimum value and a predetermined maximum value of the first objective function;

identifying a plurality of distinct and non-overlapping segments of hyperparameter values for the first objective function based on the range;

associating each of the plurality of distinct and non-overlapping segments values to a distinct one of the plurality of epsilon constraint values.

6. The system according to claim 4 , wherein the remote tuning service further:

sets one or more of the plurality of epsilon constraint values based on one or more failure regions of the second objective function, wherein the one or more failure regions includes one or more regions of hyperparameter values in which the second objective function is degraded.

7. The system according to claim 5 , wherein the remote tuning service further:

sets one or more of the plurality of epsilon constraint values based on one or more failure regions of the first objective function, wherein the one or more failure regions includes one or more regions of hyperparameter values in which the first objective function is degraded.

8. The system according to claim 4 , wherein the remote tuning service further:

implements a first tuning operation, wherein the first tuning operation includes:

progressively setting each of the plurality of epsilon constraint values; and

defining a distinct search region based on each of the plurality of epsilon constraint values; and

searching each of the distinct search regions for hyperparameter values that optimize the first objective function.

9. The system according to claim 4 , wherein the remote tuning service further:

implements a first tuning operation, wherein the first tuning operation includes:

setting a first epsilon constraint value constraining a hyperparameter search region of the first objective function;

defining the hyperparameter search region and a hyperparameter failure region;

identifying one or more possible hyperparameter values for the first objective function based on a search of the hyperparameter search region; and

blocking a search of the hyperparameter failure region.

10. The system according to claim 1 , wherein the remote tuning service further:

implements a first tuning phase of the first conditionally constrained joint function and the second conditionally constrained joint function including:

setting bounding parameters including bounding values for each of the first conditionally constrained joint function and the second conditionally constrained joint function; and

generating a random distribution of possible hyperparameter values for each of the first conditionally constrained joint function and the second conditionally constrained joint function based on the bounding parameters.

11. The system according to claim 10 , wherein the remote tuning service further:

approximates a best fit curve through the random distribution of possible hyperparameter values of at least the first conditionally constrained joint function.

12. The system according to claim 1 , further comprising:

an intelligent application programming interface that is in operable communication with the remote tuning service and that:

configures the multi-criteria tuning request, wherein configuring the multi-criteria tuning request includes:

defining the first objective function of the model, and

defining the second objective function of the model.

13. The system according to claim 12 , wherein the remote tuning service further:

selects the set of hyperparameter values along the non-convex Pareto optimal curve for the first conditionally constrained joint function; and

returns, via the intelligent application programming interface, the set of hyperparameter values to the subscriber.

14. The system according to claim 12 , wherein the remote tuning service further:

selects the set of hyperparameter values along the non-convex Pareto optimal curve for the second conditionally constrained joint function; and

returns, via an intelligent application programming interface, the set of hyperparameter values to the subscriber.

15. The system according to claim 1 , wherein

the model of the subscriber comprises a machine learning model,

the machine learning model is implemented with the set of hyperparameter values that jointly improves performances of the first objective function and the second objective function of the machine learning model.

16. A method for tuning hyperparameters for improving an effectiveness including one or more objective performance metrics of a machine learning model, the method comprising:

a remote tuning service for tuning hyperparameters of a model of a subscriber to the remote tuning service, wherein the remote tuning service is implemented by a distributed network of computers that:

receives a multi-criteria tuning work request for tuning hyperparameters of the model of the subscriber to the remote tuning service, wherein the multi-criteria tuning work request includes at least:

(i) a first objective function of the model to be optimized by the remote tuning service;

(ii) a second objective function to be optimized by the remote tuning service, the second objective function being distinct from the first objective function; computes:

a first conditionally constrained joint function for the model based on subjecting the first objective function to the second objective function;

a second conditionally constrained joint function for the model based on subjecting the second objective function to the first objective function of the model;

executes a tuning operation of the hyperparameters for the model based on a tuning of the first conditionally constrained joint function and the second conditionally constrained joint function;

simultaneously optimizes the first objective function of the model and the second objective function of the model during the tuning operation; and

identifies a set of proposed hyperparameter values that simultaneously optimizes both of the first objective function of the model and the second objective function of the model based on a hyperparameter-based point along a non-convex Pareto optimal curve; and

derives a plurality of epsilon constraint values for the first conditionally constrained function based on the second objective function, wherein the deriving includes:

identifying a range of possible hyperparameter values for the second objective function based on a predetermined minimum value and a predetermined maximum value of the second objective function;

identifying a plurality of distinct and non-overlapping segments of hyperparameter values for the second objective function based on the range;

associating each of the plurality of distinct and non-overlapping segments values to a distinct one of the plurality of epsilon constraint values.

17. The method according to claim 16 , wherein the remote tuning service further:

implements a first tuning operation, wherein the first tuning operation includes:

setting a first epsilon constraint value constraining a hyperparameter search region of the first objective function;

defining the hyperparameter search region and a hyperparameter failure region;

identifying one or more possible hyperparameter values for the first objective function based on a search of the hyperparameter search region; and

blocking a search of the hyperparameter failure region.

18. A system for tuning hyperparameters for improving an effectiveness including one or more objective performance metrics of a model, the system comprising:

a remote tuning service for tuning hyperparameters of a model of a subscriber to the remote tuning service, wherein the remote tuning service is hosted on a distributed network of computers that:

receives a multi-criteria tuning work request for tuning hyperparameters of the model of the subscriber to the remote tuning service, wherein the multi-criteria tuning work request includes at least:

(i) a first objective function of the model to be optimized by the remote tuning service;

(ii) a second objective function to be optimized by the remote tuning service, the second objective function being distinct from the first objective function; computes:

a first conditionally constrained joint function for the model based on subjecting the first objective function to the second objective function;

a second conditionally constrained joint function for the model based on subjecting the second objective function to the first objective function of the model;

executes a tuning operation of the hyperparameters for the model based on a tuning of the first conditionally constrained joint function and the second conditionally constrained joint function;

implements a first tuning phase of the first conditionally constrained joint function and the second conditionally constrained joint function including:

setting bounding parameters including bounding values for each of the first conditionally constrained joint function and the second conditionally constrained joint function; and

generating a random distribution of possible hyperparameter values for each of the first conditionally constrained joint function and the second conditionally constrained joint function based on the bounding parameters;

approximates a best fit curve through the random distribution of possible hyperparameter values of at least the first conditionally constrained joint function;

identifies that the best fit curve comprises a non-convex curve;

switches a tuning mode of the remote tuning service from a first standard tuning mode to a second non-convex tuning mode for optimizing the first objective function and the second objective function of the multi-criteria tuning request based on an epsilon constraint method; and

identifies one or more proposed hyperparameter values based on one or more hyperparameter-based points along a non-convex Pareto optimal curve.