CONSTRAINED OPTIMIZATION USING AN ANALOG PROCESSOR

1 . A method of using a hybrid analog-digital processor to optimize a system for an objective under at least one constraint, the hybrid analog-digital analog processor comprising a digital controller and an analog processor, the method comprising:

using the hybrid analog-digital processor to perform:

obtaining an objective function associated with the objective, the objective function relating sets of parameter values of the system to values providing a measure of performance of the system; and

optimizing parameters of the system, the optimizing comprising:

determining, using the analog processor, a parameter gradient for parameter values of the system based on the objective function and the at least one constraint; and

updating the parameter values of the system using the parameter gradient.

2 . The method of claim 1 , wherein determining, using the analog processor, the parameter gradient for the parameter values based on the objective function and the at least one constraint comprises:

determining, using the analog processor, a plurality of outputs of the system when configured with the parameter values; and

determining, using the analog processor, the parameter gradient using the plurality of outputs of the system configured with the parameter values.

3 . The method of claim 1 , wherein determining, using the analog processor, the parameter gradient for the parameter values based on the objective function and the at least one constraint comprises:

performing, using the analog processor, at least one matrix operation to obtain at least one output of the at least one matrix operation; and

determining the parameter gradient using the at least one output of the at least one matrix operation.

4 . The method of claim 3 , wherein performing, using the analog processor, the at least one matrix operation comprises:

determining a scaling factor for a portion of a matrix involved in the at least one matrix operation;

scaling the portion of the matrix using the scaling factor to obtain a scaled portion of the matrix;

programming the analog processor using the scaled portion of the matrix; and

performing, by the analog processor programmed using the scaled the portion of the matrix, the at least one matrix operation to obtain the at least one output of the at least one matrix operation.

5 . The method of claim 1 , wherein the at least one constraint comprises at least one constraint function and the method further comprises:

generating a combined function using the objective function and the at least one constraint function;

wherein determining, using the analog processor, the parameter gradient for the parameter values based on the objective function and the at least one constraint comprises:

determining a gradient of the combined function for the parameter values.

6 . The method of claim 1 , wherein:

the at least one constraint comprises at least one constraint function; and

determining, using the analog processor, the parameter gradient for the parameter values based on the objective function and the at least one constraint comprises:

determining a gradient of the objective function for the parameter values;

determining a gradient of the at least one constraint function for the parameter values; and

determining the parameter gradient using the gradient of the objective function and the gradient of the at least one constraint function.

7 . The method of claim 6 , wherein determining the parameter gradient using the gradient of the objective function and the gradient of the at least one constraint function comprises:

determining a normalization of the gradient of the objective function;

determining a normalization of the gradient of the at least one constraint function; and

determining the parameter gradient using normalizations of the gradient of the objective function and the gradient of the at least one constraint function.

8 . The method of claim 1 , wherein the at least one constraint comprises at least one inequality constraint.

9 . The method of claim 1 , wherein:

the at least one constraint comprises a plurality of constraints represented by a plurality of constraint functions; and

determining, using the analog processor, the parameter gradient for the parameter values comprises:

generating a barrier function using the plurality of constraint functions;

determining a gradient of the objective function for the parameter values;

determining a gradient of the barrier function for the parameter values; and

determining the parameter gradient using the gradient of the objective function and the gradient of the barrier function.

10 . The method of claim 9 , wherein generating the barrier function using the plurality of constraints comprises generating a logarithmic barrier function.

11 . The method of claim 1 , further comprising:

after optimizing the parameter values of the system, performing a subsequent optimization on the parameter values of the system using a digital processor.

12 . The method of claim 1 , wherein the system is a machine learning system, and the objective is a task to be performed by the machine learning system.

13 . An optimization system for optimizing a system for an objective under at least one constraint, the optimization system comprising:

a hybrid analog-digital processor comprising a digital controller and an analog processor, the hybrid analog-digital processor configured to:

obtain an objective function associated with the objective, the objective function relating sets of parameter values of the system to values providing a measure of performance of the system; and

optimize parameters of the system, the optimizing comprising:

determining, using the analog processor, a parameter gradient for parameter values of the system based on the objective function and the at least one constraint; and

updating the parameter values of the system using the parameter gradient.

14 . The optimization system of claim 13 , wherein determining, using the analog processor, the parameter gradient for the parameter values based on the objective function and the at least one constraint comprises:

determining, using the analog processor, a plurality of outputs of the system when configured with the parameter values; and

determining, using the analog processor, the parameter gradient using the plurality of outputs of the system configured with the parameter values.

15 . The optimization system of claim 13 , wherein determining, using the analog processor, the parameter gradient for the parameter values based on the objective function and the at least one constraint comprises:

performing, using the analog processor, at least one matrix operation to obtain at least one output of the at least one matrix operation; and

determining the parameter gradient using the at least one output of the at least one matrix operation.

16 . The optimization system of claim 15 , wherein performing, using the analog processor, the at least one matrix operation comprises:

determining a scaling factor for a portion of a matrix involved in the at least one matrix operation;

scaling the portion of the matrix using the scaling factor to obtain a scaled portion of the matrix;

programming the analog processor using the scaled portion of the matrix; and

performing, by the analog processor programmed using the scaled the portion of the matrix, the at least one matrix operation to obtain the at least one output of the at least one matrix operation.

17 . The optimization system of claim 13 , wherein the at least one constraint comprises at least one constraint function and the hybrid analog-digital processor is further configured to:

generate a combined function using the objective function and the at least one constraint function;

wherein determining, using the analog processor, the parameter gradient for the parameter values based on the objective function and the at least one constraint comprises:

determining a gradient of the combined function for the parameter values.

18 . The optimization system of claim 13 , wherein:

the at least one constraint comprises at least one constraint function; and

determining, using the analog processor, the parameter gradient for the parameter values based on the objective function and the at least one constraint comprises:

determining a gradient of the objective function for the parameter values;

determining a gradient of the at least one constraint function for the parameter values; and

determining the parameter gradient using the gradient of the objective function and the gradient of the at least one constraint function.

19 . The optimization system of claim 13 , wherein:

the at least one constraint comprises a plurality of constraints represented by a plurality of constraint functions; and

determining, using the analog processor, the parameter gradient for the parameter values comprises:

generating a barrier function using the plurality of constraint functions;

determining a gradient of the objective function for the parameter values;

determining a gradient of the barrier function for the parameter values; and

determining the parameter gradient using the gradient of the objective function and the gradient of the barrier function.

20 . A non-transitory computer-readable storage medium storing instructions that, when executed by a hybrid analog-digital processor comprising a digital controller and an analog processor, cause the hybrid analog-digital processor to perform a method of optimizing a system for an objective under at least one constraint, the method comprising:

obtaining an objective function associated with the objective, the objective function relating sets of parameter values of the system to values providing a measure of performance of the system; and

optimizing parameters of the system, the optimizing comprising:

determining, using the analog processor, a parameter gradient for parameter values of the system based on the objective function and the at least one constraint; and

updating the parameter values of the system using the parameter gradient.