Machine learning device, servo control device, servo control system, and machine learning method

1. A machine learning device configured to perform machine learning with respect to a plurality of servo control units configured to control a plurality of motors, the motors configured to drive a machine having a plurality of axes, with one axis among the plurality of axes receiving interference generated by movement along at least one of the other axes,

a first servo control unit related to the one axis receiving the interference among the plurality of servo control units, the first servo control unit comprising:

compensation units including a position error compensation unit configured to obtain a compensation value for compensating for a position error, a velocity command compensation unit configured to obtain a compensation value for compensating for a velocity command, and a torque command compensation unit configured to obtain a value for compensating for a torque command of the first servo control unit based on one or more functions including at least one of variables related to a position command and variables related to position feedback information of a second servo control unit related to the axis generating the interference, and

the machine learning device comprising:

a state information acquisition unit configured to acquire state information including first servo control information of the first servo control unit, second servo control information of the second servo control unit, and coefficients for each of the position error compensation unit, the velocity command compensation unit, and the torque command compensation unit based on the one or more functions;

an action information output unit configured to output action information including adjustment information for adjusting of the coefficients for each of the position error compensation unit, the velocity command compensation unit, and the torque command compensation unit of the compensation units included in the state information to the compensation units;

a reward output unit configured to output a reward value each time an action is executed based on the adjusting of the coefficients for reinforcement learning using an evaluation function serving as a function of the first servo control information; and

a value function updating unit configured to update a value function related to the adjustment information of the coefficients based on the reward value output by the reward output unit, the state information, and the action information.

2. The machine learning device according to claim 1 , wherein

the first servo control information includes a position command and position feedback information of the first servo control unit or a first position error of the first servo control unit, and

the evaluation function outputs the reward value based on a value including a second position error obtained from the position command and the position feedback information of the first servo control unit or the first position error, an absolute value of the first or second position error, or a second power of the absolute value.

3. The machine learning device according to claim 1 , wherein

the variable related to the position command of the second servo control unit is at least one of the position command, a differentiated value of the position command, and a double-differentiated value of the position command of the second servo control unit, and

the variable related to the position feedback information of the second servo control unit is at least one of the position feedback information, a differentiated value of the position feedback information, and a double-differentiated value of the position feedback information of the second servo control unit.

4. The machine learning device according to claim 1 , wherein

a machining program during learning for controlling the first and second servo control units makes the first and second servo control units perform movement along the axis generating the interference and stop movement along the axis receiving the interference during the machine learning.

5. The machine learning device according to claim 1 , the machine learning device comprising:

an optimization action information output unit configured to output the adjustment information for adjusting of the coefficients included in the state information based on the value function updated by the value function updating unit, and to output the adjustment information to the compensation units.

6. A servo control device including: the machine learning device according to claim 1 ; and

the plurality of servo control units configured to control the plurality of motors, the motors configured to drive the machine having the plurality of axes, with the one axis among the plurality of axes receiving the interference generated by the movement along at least one of the other axes, wherein the first servo control unit related to the one axis receiving the interference among the plurality of servo control units comprises the compensation units configured to obtain the corresponding compensation value for compensating for the at least one of the position error, the velocity command, and the torque command of the first servo control unit based on the one or more functions including the at least one of the variable related to the position command and the variable related to the position feedback information of the second servo control unit related to the axis generating the interference, and wherein

the machine learning device outputs the action information including the adjustment information of the coefficients to the compensation units.

7. A servo control system including:

the machine learning device according to claim 1 ; and

a servo control device including the plurality of servo control units configured to control the plurality of motors, the motors configured to drive the machine having the plurality of axes, with the one axis among the plurality of axes receiving the interference generated by the movement along at least one of the other axes, wherein

the first servo control unit related to the one axis receiving the interference among the plurality of servo control units comprises the compensation units configured to obtain the corresponding compensation value for compensating for the at least one of a position error, the velocity command, and the torque command of the first servo control unit based on the one or more functions including the at least one of the variable related to a position command and the variable related to the position feedback information of the second servo control unit related to the axis generating the interference, and wherein

the machine learning device outputs the action information including the adjustment information of the coefficients to the compensation units.

8. A machine learning method for a machine learning device configured to perform machine learning with respect to a plurality of servo control units configured to control a plurality of motors, the motors configured to drive a machine having a plurality of axes, with one axis among the plurality of axes receiving interference generated by movement along at least one of the other axes,

a first servo control unit related to the one axis receiving the interference among the plurality of servo control units, the first servo control unit comprising compensation units including a position error compensation unit configured to obtain a compensation value for compensating for a position error, a velocity command compensation unit configured to obtain a compensation value for compensating for a velocity command, and a torque command compensation unit configured to obtain a compensation value for compensating for a torque command of the first servo control unit based on one or more functions including at least one of variables related to a position command and variables related to position feedback information of a second servo control unit related to the axis generating the interference, and

the machine learning method comprising the steps of:

acquiring state information including first servo control information of the first servo control unit, second servo control information of the second servo control unit, and coefficients for each of the position error compensation unit, the velocity command compensation unit, and the torque command compensation unit based on the one or more functions;

outputting action information including adjustment information for adjusting of the coefficients for each of the position error compensation unit, the velocity command compensation unit, and the torque command compensation unit of the compensation units included in the state information to the compensation units;

outputting a reward value each time an action is executed based on the adjusting of the coefficients for reinforcement learning using an evaluation function serving as a function of the first servo control information; and

updating a value function related to the adjustment information of the coefficients based on the reward value, the state information, and the action information.

9. The machine learning method according to claim 8 , wherein the machine learning device outputs the adjustment information for adjusting of the coefficients included in the state information based on the updated value function, and outputs the adjustment information to the compensation units.