Machine learning device, robot controller, robot system, and machine learning method for learning action pattern of human
A machine learning device for a robot that allows a human and the robot to work cooperatively, the machine learning device including a state observation unit that observes a state variable representing a state of the robot during a period in that the human and the robot work cooperatively; a determination data obtaining unit that obtains determination data for at least one of a level of burden on the human and a working efficiency; and a learning unit that learns a training data set for setting an action of the robot, based on the state variable and the determination data.
1. A machine learning device for a robot configured to allow a human and the robot to work cooperatively, the machine learning device comprising:
at least one memory; and
at least one processor configured to:
obtain a state variable representing a state of the robot when the human and the robot work cooperatively,
obtain determination data for a level of burden on the human, wherein the level of burden varies based on an action of the robot,
set a reward based on the determination data, and
update a neural network representing a function for setting the action of the robot, based on the state variable and the reward,
wherein the at least one processor is configured to
set the reward to become greater for a smaller value of the level of burden, and
update the neural network to set the action of the robot that leads to a smaller level of burden on the human.
2. The machine learning device according to claim 1 , wherein the state variable comprises at least one of a position, an orientation, a velocity, and an acceleration of the robot.
3. The machine learning device according to claim 1 , wherein the determination data is calculated based on a force from the human when the human and the robot work cooperatively.
4. The machine learning device according to claim 1 , wherein the determination data comprises at least one of a magnitude and a direction of a load sensed by the robot, a magnitude and a direction of a load sensed by an environment surrounding the robot, a level of burden on the environment surrounding the robot, and a time taken to move the robot.
5. The machine learning device according to claim 1 , wherein the action of the robot comprises at least one of a position, an orientation, a velocity, and an acceleration of the robot when the robot moves.
6. The machine learning device according to claim 1 , wherein the neural network represents the function of predetermined respective positions of the robot when the robot moves.
7. The machine learning device according to claim 1 , wherein the at least one processor is further configured to update the neural network based on an identified result of the human.
8. The machine learning device according to claim 1 , wherein information obtained by learning in a second robot, which is different from the robot, based on a state variable and a reward for the second robot is shared for the robot and the second robot.
9. The machine learning device according to claim 1 , wherein the at least one processor is further configured to set a greater reward for a smaller absolute value of an acceleration of the robot, and a greater reward for a shorter time taken to move the robot.
10. The machine learning device according to claim 1 , wherein the at least one processor is further configured to
identify the human who works cooperatively with the robot, and
update the neural network representing the function for setting the action of the robot, based on the identified human.
11. The machine learning device according to claim 1 , wherein the robot comprises one of an industrial robot, a field robot, and a service robot.
12. A robot system comprising:
the machine learning device according to claim 1 ; and
the robot configured to work cooperatively with the human, wherein the at least one processor is further configured to control an action of the robot, and the action of the robot is set based on an output of the neural network.
13. The robot system according to claim 12 , wherein the machine learning device is set on a cloud server.
14. The robot system according to claim 12 , wherein
the robot comprises:
a first sensor configured to output a signal corresponding to a force from the human; and
a second sensor configured to detect a position and an orientation of the robot, wherein the at least one processor is further configured to
obtain the determination data, based on output of the first sensor, and
obtain the state variable, based on output of the second sensor.
15. The robot system according to claim 14 , wherein the second sensor comprises at least one of a motion sensor, a pressure sensor, a torque sensor for a motor, and a contact sensor.
16. A robot system comprising:
a second robot configured to work cooperatively with a second human; and
the neural network updated by the machine learning device according to claim 1 , wherein the at least one processor is further configured to
control an action of the second robot, and
obtain a state variable representing a state of the second robot when the second human and the second robot work cooperatively; and
the action of the second robot is set based on an output of the neural network.
17. The machine learning device according to claim 1 , wherein
the determination data further includes information for a working efficiency, and
the at least one processor is configured to
set the reward to become greater for a higher value for the working efficiency, and
update the neural network to set the action of the robot that leads to a higher value for the working efficiency.
18. The machine learning device according to claim 1 , wherein the level of burden on the human is determined based on at least one of a velocity of the robot and a distance of the robot from the human.
19. A machine learning method, comprising:
obtaining a state variable representing a state of a robot when a human and the robot work cooperatively;
obtaining determination data for a level of burden on the human, wherein the level of burden varies based on an action of the robot;
setting a reward based on the determination data; and
updating a neural network representing a function for setting the action of the robot, based on the state variable and the reward,
wherein the reward becomes greater for a smaller value of the level of burden, and the neural network is updated to set the action of the robot that leads to a smaller level of burden on the human.
20. The method according to claim 19 , wherein the state variable comprises at least one of a position, an orientation, a velocity, and an acceleration of the robot.
21. The method according to claim 19 , wherein the determination data is calculated based on a force from the human when the human and the robot work cooperatively.
22. The method according to claim 19 , wherein the action of the robot comprises at least one of a position, an orientation, a velocity, and an acceleration of the robot when the robot moves.
23. A robot, comprising:
at least one memory; and
at least one processor configured to:
obtain a state variable representing a state of the robot when a human and the robot work cooperatively,
obtain an output from a neural network having the state variable as input information, and
control, based on the output from the neural network, an action of the robot when the human and the robot work cooperatively,
wherein the neural network has been updated, based on a reward for burden on another human, to set the action of the robot that leads to a smaller burden on the human.