Drilling control

1 . A method comprising:

generating an avatar comprising a machine learning model, the machine learning model comprising a controller;

training an agent using a reward function customizable for desired drilling behavior;

issuing control instructions by the controller to control rig equipment for drilling of a borehole in a subsurface environment, wherein the controller comprises a tunable weight-on-bit set point control loop;

during the drilling of the borehole in the subsurface environment, receiving sensor data as feedback;

dynamically determining a tuning parameter value for the tunable weight-on-bit set point control loop using the feedback and the agent;

providing, via the agent, the tuning parameter value for the tunable weight-on-bit set point control loop to the controller;

determining a control instruction using the controller and the tuning parameter value for the tunable weight-on-bit set point control loop in the tunable weight-on-bit set point control loop; and

issuing the control instruction by the controller to control the rig equipment for drilling of the borehole according to the desired drilling behavior.

2 . The method of claim 1 , wherein the controller comprises a tunable torque set point control loop and the method comprises dynamically determining a tuning parameter value for the tunable torque set point control loop using the feedback and the agent.

3 . The method of claim 2 , comprising determining the control instruction using the controller and the tuning parameter value for the tunable torque set point control loop in the tunable torque set point control loop.

4 . The method of claim 2 , wherein the controller comprises a tunable differential pressure set point control loop and the method comprises dynamically determining a tuning parameter value for the tunable differential pressure set point control loop using the feedback and the agent.

5 . The method of claim 4 , comprising:

determining the control instruction using the controller and the tuning parameter value for the tunable differential pressure set point control loop in the tunable torque set point control loop; and

issuing the control instruction by the controller to control the rig equipment for the drilling of the borehole according to the desired drilling behavior.

6 . The method of claim 1 , wherein the tuning parameter value for the tunable weight-on-bit set point control loop is a gain.

7 . The method of claim 6 , wherein the tunable weight-on-bit set point control loop comprises one or more of proportional control and integral control and wherein the gain is correspondingly one or more of a proportional control gain and an integral control gain.

8 . The method of claim 1 , wherein the reward function customizable for the desired drilling behavior comprises a number of terms that comprise adjustable constants.

9 . The method of claim 1 , wherein the reward function customizable for the desired drilling behavior comprises one or more of a drilling speed term, a term related to a smoothness of the drilling of the borehole in the subsurface environment, and a constraint term.

10 . The method of claim 9 , wherein the drilling speed term rewards a higher rate of penetration.

11 . The method of claim 10 , wherein the drilling speed term comprises a ratio of an average rate of penetration and a rate of penetration limit.

12 . The method of claim 9 , wherein the term related to the smoothness of the drilling of the borehole in the subsurface environment is directly related to a magnitude of fluctuations in one or more of rate of penetration, weight-on-bit, torque, or differential pressure.

13 . The method of claim 12 , wherein the term related to the smoothness of the drilling of the borehole in the subsurface environment comprises one or more of a rate of penetration standard deviation, a weight-on-bit standard deviation, a torque standard deviation, and a differential pressure standard deviation.

14 . The method of claim 1 , wherein the reward function customizable for the desired drilling behavior comprises one or more time-dependent penalty terms.

15 . The method of claim 14 , wherein the one or more time-dependent penalty terms comprise one or more of a time that rate of penetration is above a corresponding rate of penetration set point, a time that weight-on-bit is above a corresponding weight-on-bit set point, a time that torque is above a corresponding torque set point, or a time that differential pressure is above a corresponding differential pressure set point.

16 . The method of claim 1 , wherein the reward function customizable for the desired drilling behavior comprises at least one positive reward term and at least one negative penalty term.

17 . The method of claim 1 , wherein the reward function customizable for the desired drilling behavior depends on a measured depth of the borehole during a bit run to disfavor premature bit wear early in the bit run and to favor drilling speed later in the bit run.

18 . A system comprising:

a processor;

memory accessible to the processor; and

processor-executable instructions stored in the memory and executable by the processor to instruct the system to:

generate an avatar comprising a machine learning model, the machine learning model comprising a controller;

train an agent using a reward function customizable for desired drilling behavior;

issue control instructions by the controller to control rig equipment for drilling of a borehole in a subsurface environment, wherein the controller comprises a tunable weight-on-bit set point control loop;

during the drilling of the borehole, receive sensor data as feedback;

dynamically determine a tuning parameter value for the tunable weight-on-bit set point control loop using the feedback and the agent;

provide, via the agent, the tuning parameter value for the tunable weight-on-bit set point control loop to the controller;

determine a control instruction using the tuning parameter value for the tunable weight-on-bit set point control loop in the tunable weight-on-bit set point control loop; and

issue the control instruction by the controller to control the rig equipment for drilling of the borehole according to the desired drilling behavior.

19 . One or more computer-readable storage media comprising computer-executable instructions executable to instruct a computing system to:

generate an avatar comprising a machine learning model, the machine learning model comprising a controller;

train an agent using a reward function customizable for desired drilling behavior;

issue control instructions by the controller to control rig equipment for drilling of a borehole in a subsurface environment, wherein the controller comprises a tunable weight-on-bit set point control loop;

during the drilling of the borehole in the subsurface environment, receive sensor data as feedback;

dynamically determine a tuning parameter value for the tunable weight-on-bit set point control loop using the feedback and the agent;

provide, via the agent, the tuning parameter value for the tunable weight-on-bit set point control loop to the controller;

determine a control instruction using the tuning parameter value for the tunable weight-on-bit set point control loop in the tunable weight-on-bit set point control loop; and

issue the control instruction by the controller to control the rig equipment for drilling of the borehole according to the desired drilling behavior.

20 . The method of claim 1 , wherein the machine learning model comprises a hybrid machine learning model including the controller and a plant model configured to output a physical response of a drilling system.