Controlled adaptive optimization
Generally, the present disclosure is directed to systems and methods that perform adaptive optimization with improved convergence properties. The adaptive optimization techniques described herein are useful in various optimization scenarios, including, for example, training a machine-learned model such as, for example, a neural network. In particular, according to one aspect of the present disclosure, a system implementing the adaptive optimization technique can, over a plurality of iterations, employ an adaptive effective learning rate while also ensuring that the effective learning rate is non-increasing.
1. A computer-implemented method for optimizing machine-learned models that provides improved convergence properties, the method comprising:
for each of a plurality of iterations:
determining, by one or more computing devices, a gradient of a loss function that evaluates a performance of a machine-learned model that comprises a plurality of parameters;
determining, by the one or more computing devices, a current learning rate control value based on the gradient of the loss function, wherein the current learning rate control value equals a most recent learning rate control value minus an update value, wherein a magnitude of the update value is a function of the gradient of the loss function but not the most recent learning rate control value, and wherein a polarity of the update value is a function of both the gradient of the loss function and the most recent learning rate control value;
determining, by the one or more computing devices, a current effective learning rate based at least in part on the current learning rate control value; and
determining, by the one or more computing devices, an updated set of values for the plurality of parameters of the machine-learned model based at least in part on the gradient of the loss function and according to the current effective learning rate; and
providing, by the one or more computing devices, an optimized version of the machine-learned model as an output, the optimized version of the machine-learned model comprising a final set of values for the plurality of parameters;
wherein, for at least one of the plurality of iterations, the polarity of the update value is positive such that the current learning rate control value is less than the most recent learning rate control value, whereby the current effective learning rate is greater than a most recent effective learning rate.
2. The computer-implemented method of claim 1 , wherein the update value is equal to a square of the gradient of the loss function multiplied by a sign function applied to the most recent learning rate control value minus the square of the gradient of the loss function and multiplied by a scaling coefficient that is equal to one minus an update scaling parameter.
3. The computer-implemented method of claim 1 , wherein, over the plurality of iterations, the update scaling parameter is held constant.
4. The computer-implemented method of claim 1 , wherein, over the plurality of iterations, the update scaling parameter is increased so as to provide increasing influence to past learning rate control values.
5. The computer-implemented method of claim 1 , wherein determining, by the one or more computing devices, the updated set of values for the plurality of parameters of the machine-learned model based at least in part on the gradient of the loss function and according to the current effective learning rate comprises:
updating, by the one or more computing devices, a current momentum value based at least in part on the gradient of the loss function and one or more previous momentum values respectively from one or more previous iterations; and
determining, by the one or more computing devices, the updated set of values for the plurality of parameters of the machine-learned model based at least in part on the current momentum value and according to the current effective learning rate.
6. The computer-implemented method of claim 1 , wherein determining, by the one or more computing devices, the current effective learning rate based at least in part on the current learning rate control value comprises dividing, by the one or more computing devices, a current learning rate by a square root of the current learning rate control value.
7. The computer-implemented method of claim 1 , wherein determining, by the one or more computing devices, the current effective learning rate based at least in part on the current learning rate control value comprises dividing, by the one or more computing devices, a current learning rate by a square root of the current learning rate control value plus an adaptivity control value.
8. A computing system, comprising:
one or more processors; and
one or more non-transitory computer-readable media that store instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, the operations comprising, for each of one or more iterations:
determining a gradient of a loss function that evaluates a performance of a machine-learned model that comprises a plurality of parameters;
determining a current learning rate control value based on the gradient of the loss function, wherein the current learning rate control value equals a most recent learning rate control value minus an update value, wherein a magnitude of the update value is equal to a square of the gradient of the loss function times a scaling coefficient, and wherein a polarity of the update value is a function of both the gradient of the loss function and the most recent learning rate control value;
determining a current effective learning rate based at least in part on the current learning rate control value; and
determining an updated set of values for the plurality of parameters of the machine-learned model based at least in part on the gradient of the loss function and according to the current effective learning rate.
9. The computing system of claim 8 , wherein the update value is equal to the square of the gradient of the loss function multiplied by a sign function applied to the most recent learning rate control value minus the square of the gradient of the loss function and multiplied by the scaling coefficient, wherein the scaling coefficient is equal to one minus an update scaling parameter.
10. The computing system of claim 8 , wherein the one or more iterations comprise a plurality of iterations and wherein, for at least one of the plurality of iterations, the polarity of the update value is positive such that the current learning rate control value is less than the most recent learning rate control value, whereby the current effective learning rate is greater than a most recent effective learning rate.
11. The computing system of claim 8 wherein, over the one or more iterations, the update scaling parameter is held constant.
12. The computing system of claim 8 , wherein, over the one or more iterations, the update scaling parameter is increased so as to provide increasing influence to past learning rate control values.
13. The computing system of claim 8 , wherein determining, by the one or more computing devices, the updated set of values for the plurality of parameters of the machine-learned model based at least in part on the gradient of the loss function and according to the current effective learning rate comprises:
updating, by the one or more computing devices, a current momentum value based at least in part on the gradient of the loss function and one or more previous momentum values respectively from one or more previous iterations; and
determining, by the one or more computing devices, the updated set of values for the plurality of parameters of the machine-learned model based at least in part on the current momentum value and according to the current effective learning rate.
14. The computing system of claim 8 , wherein determining, by the one or more computing devices, the current effective learning rate based at least in part on the current learning rate control value comprises dividing, by the one or more computing devices, a current learning rate by a square root of the current learning rate control value.
15. The computing system of claim 8 , wherein determining, by the one or more computing devices, the current effective learning rate based at least in part on the current learning rate control value comprises dividing, by the one or more computing devices, a current learning rate by a square root of the current learning rate control value plus an adaptivity control value.
16. One or more non-transitory computer-readable media that store instructions that, when executed by one or more processors, cause the one or more processors to perform operations, the operations comprising:
for each of a plurality of iterations:
determining a gradient of a loss function that evaluates a performance of a machine-learned model that comprises a plurality of parameters;
determining a current learning rate control value based on the gradient of the loss function, wherein the current learning rate control value equals a most recent learning rate control value minus an update value, wherein the update value is equal to a square of the gradient of the loss function multiplied by a sign function applied to the most recent learning rate control value minus the square of the gradient of the loss function and multiplied by a scaling coefficient that is equal to one minus an update scaling parameter;
determining, by the one or more computing devices, a current effective learning rate based at least in part on the current learning rate control value; and
updating at least one of the plurality of parameters of the machine-learned model based at least in part on the gradient of the loss function and according to a current effective learning rate that is a function of the current learning rate control value.
17. The one or more non-transitory computer-readable media of claim 16 , wherein the current effective learning rate is inversely correlated to the current learning rate control value.
18. The one or more non-transitory computer-readable media of claim 16 , wherein the one or more iterations comprise a plurality of iterations and wherein, for at least one of the plurality of iterations, the polarity of the update value is positive such that the current learning rate control value is less than the most recent learning rate control value, whereby the current effective learning rate is greater than a most recent effective learning rate.
19. The one or more non-transitory computer-readable media of claim 16 , wherein, over the one or more iterations, the update scaling parameter is held constant or increased.