Application of simple random search approach for reinforcement learning to controller tuning parameters

1. A method for reinforcement learning to automate tuning of a controller to control a process in an industrial plant, comprising:

supplying, by a summation unit, an output of a controller to an industrial plant in a closed loop structure;

processing a policy update by feeding an output of the industrial plant to a total reward through a memory of the closed loop structure;

providing a perturb policy based on an output of the policy update;

varying parameters of the policy update to provide a finite-difference approach to tune the controller from a closed-loop step-response using the closed loop structure;

applying the finite-difference approach to tune the controller; and

tuning the controller in response to applying the finite-difference approach by taking a state as an entirety of the closed-loop step response.

2. The method of claim 1 wherein the controller comprises a PID (Proportional Integral Derivative) controller.

3. The method of claim 2 wherein the policy comprises a function from a step-response to new PID gains.

4. The method of claim 2 wherein tuning the controller further comprises changing a set point.

5. The method of claim 2 wherein tuning the controller further comprises:

varying parameters of a policy, wherein the policy comprises a function from a step-response to new PID gains; and

changing a set point.

6. The method of claim 1 wherein tuning the controller with respect to the entirety of the closed-loop step response facilitates iteratively improving a gain toward a desired closed-loop response to allow for prescribing a stability requirement into a reward function without a modeling procedure.

7. The method of claim 1 wherein the controller is tuned under a condition of plant uncertainty.

8. A system for reinforcement learning to automate tuning of a controller to control a process in an industrial plant, comprising:

at least one processor; and

a non-transitory computer-usable medium embodying computer program code, said computer-usable medium capable of communicating with said at least one processor, said computer program code comprising instructions executable by said at least one processor and configured for:

supplying, by a summation unit, output of a controller to a plant in a closed loop structure;

processing a policy update by feeding an output of the plant to a total reward through a memory of the closed loop structure;

providing a perturb policy based on the output of the policy update;

varying parameters of the policy update to provide a finite-difference approach to tune the controller from a closed-loop step-response using the closed loop structure;

applying the finite-difference approach to tune the controller; and

tuning the controller in response to applying the finite-difference approach by taking a state as an entirety of the closed-loop step response.

9. The system of claim 8 wherein the controller comprises a PID (Proportional Integral Derivative) controller.

10. The system of claim 9 wherein the policy comprises a function from a step-response to new PID gains.

11. The system of claim 9 wherein the instructions for tuning the controller further comprises instructions configured for changing a set point.

12. The system of claim 9 wherein the instructions for tuning the controller further comprise instructions configured for:

varying parameters of a policy, wherein the policy comprises a function from a step-response to new PID gains; and

changing a set point.

13. The system of claim 8 wherein the instructions for tuning the controller with respect to the entirety of the closed-loop step response facilitates iteratively improving a gain toward a desired closed-loop response to allow for prescribing a stability requirement into a reward function without a modeling procedure.

14. The system of claim 8 wherein the controller is tuned under a condition of plant uncertainty.

15. An apparatus for reinforcement learning to automate tuning of a controller to control a process in an industrial plant, comprising:

at least one processing device configured to:

supply, by a summation unit, an output of a controller to an industrial plant in a closed loop structure

process a policy update by feeding an output of the industrial plant to a total reward through a memory of the closed loop structure;

provide perturb policy based on an output of the policy update; and

vary parameters of the policy update to provide a finite-difference approach to tune the controller from a closed-loop step-response using the closed loop structure,

wherein the finite-difference approach is applied to tuning the controller and the controller is tuned in response to applying the finite-difference approach by taking a state as an entirety of the closed-loop step response.

16. The apparatus of claim 15 wherein the controller comprises a PID (Proportional Integral Derivative) controller.

17. The apparatus of claim 16 wherein the policy comprises a function from a step-response to new PID gains.

18. The apparatus of claim 16 wherein the controller is tuned by changing a set point.

19. The apparatus of claim 16 wherein the controller is tuned by:

varying parameters of a policy, wherein the policy comprises a function from a step-response to new PID gains; and

changing a set point.

20. The apparatus of claim 15 wherein tuning the controller with respect to the entirety of the closed-loop step response facilitates iteratively improving a gain toward a desired closed-loop response to allow for prescribing a stability requirement into a reward function without a modeling procedure.