Control system and machine learning device
Provided are a controller and a machine learning device that perform machine learning to optimize the servo gain of a machine inside a facility in accordance with action conditions, action environments, and a priority factor of the machine. The control system observes machine information on a machine as state, acquires information on machining by a machine as determination data, calculates a reward based on the determination data and reward conditions, performs the machine learning of the adjustment of the servo gain of the machine, determines an action of adjustment of the servo gain of the machine based on the state data and a machine learning result of the adjustment of the servo gain of the machine, and changes the servo gain of the machine, based on the action of adjustment of the determined servo gain.
1. A control system having at least one machine that machines a workpiece and a high-order apparatus that adjusts servo gain used in machining by the machine, the control system comprising:
a machine learning device including processor that performs machine learning of an adjustment of the servo gain of the machine, wherein the processor is configured to:
observe machine information on the machine as state data;
acquire information on machining by the machine as determination data;
calculate a reward based on the determination data and a plurality of preset reward conditions that are defined by a priority factor preset for the machine according to preferences given in the adjustment of the servo gain;
perform the machine learning of the adjustment of the servo gain of the machine;
determine an action of adjustment of the servo gain of the machine, based on the state data and a machine learning result of the adjustment of the servo gain of the machine;
change the servo gain of the machine based on the action of adjustment of the servo gain; and
perform the machine learning of the adjustment of the servo gain of the machine, based on the state data, the action of adjustment, and the reward calculated after the action of adjustment.
2. The control system according to claim 1 ,
wherein the processor is further configured to switch a value function used in the machine learning and the determination of the action of adjustment, based on the priority factor preset for the machine.
3. The control system according to claim 1 , wherein
a positive reward or a negative reward is calculated based on the plurality of preset reward conditions set correspondingly to the priority factor preset for the machine.
4. The control system according to claim 1 , wherein
the control system is connected to at least one another high-order apparatus and mutually exchanges or shares the machine learning result with the other high-order apparatus.
5. A machine learning device that performs machine learning of an adjustment of servo gain used in machining by at least one machine that machines a workpiece, the machine learning device comprising:
a processor configured to:
observe machine information on the machine as state data;
acquire information on machining by the machine as determination data;
calculate a reward based on the determination data and a plurality of preset reward conditions that are defined by a priority factor preset for the machine according to preferences given in the adjustment of the servo gain;
perform the machine learning of the adjustment of the servo gain of the machine;
determine an action of adjustment of the servo gain of the machine, based on the state data and a machine learning result of the adjustment of the servo gain of the machine;
change the servo gain of the machine, based on the action of adjustment of the servo gain; and
perform the machine learning of the adjustment of the servo gain of the machine, based on the state data, the action of adjustment, and the reward calculated after the action of adjustment.
6. The machine learning device according to claim 5 ,
wherein the processor is further configured to switch a value function used in the machine learning and the determination of the action of adjustment, based on the priority factor preset for the machine.
7. The control system according to claim 1 , wherein the priority factor preset for the machine includes at least any one improvement in machining quality, improvement in productivity, and energy saving performance.
8. The control system according to claim 1 , wherein the plurality of preset reward conditions are used in combination according to the priority factor preset for the machine.