Robot motion planning device, robotic system, and method

1. A robot motion planning device comprising:

a memory; and

processing circuitry configured to:

receive observation information obtained by observing at least part of a movable range of a robot;

determine, in a case where first observation information is received, a first target position to which the robot is to make a motion, using an action-value function and the first observation information, the action-value function being a function of observation information and a position and being indicative of a value of a motion to be made by the robot;

receive measurement information obtained by measuring a state of the robot;

calculate a first difference corresponding to the first observation information, using the measurement information; and

determine a motion plan of a force-controlled motion of the robot, based on the first target position and the first difference,

wherein the processing circuitry is configured to make a distance between a start point and an end point of the force-controlled motion longer as the first difference is greater.

2. The robot motion planning device of claim 1 , wherein

the processing circuitry is further configured to:

determine a first reward, based on a motion made by the robot according to the motion plan; and

update the action-value function, using the first reward.

3. The robot motion planning device of claim 2 , wherein

the processing circuitry is configured to:

calculate an update amount of the action-value function, using at least the first reward and the action-value function; and

update the action-value function, using the update amount.

4. The robot motion planning device of claim 2 , wherein

the processing circuitry is further configured to:

determine second rewards corresponding to positions, respectively, based on the motion made by the robot according to the motion plan; and

update the action-value function, using the second rewards.

5. The robot motion planning device of claim 2 , wherein

the processing circuitry is further configured to determine the first target position at random from a range that is determined based on the action-value function and the first difference.

6. The robot motion planning device of claim 1 , wherein

the processing circuitry is configured to:

receive first measurement information obtained by measuring a state of the robot that makes a motion according to the motion plan; and

update the first difference, using the first measurement information.

7. The robot motion planning device of claim 6 , wherein

the processing circuitry is configured to update the first difference, using a difference between a first change point of a first action-value function of the action-value function and a second change point of the first measurement information, the first action-value function corresponding to the first observation information.

8. The robot motion planning device of claim 7 , wherein

the first measurement information comprises pieces of information, and each of the pieces of information indicates a position of a movable portion of the robot and force applied to the movable portion at the position, and

the processing circuitry is configured to update the first difference, using a difference between a first position where the first action-value function declines by a first threshold value or more and a second position where the first measurement information exceeds a second threshold value.

9. The robot motion planning device of claim 1 , wherein

the processing circuitry is further configured to display information on the first difference on a screen of a display.

10. The robot motion planning device of claim 9 , wherein

the information on the first difference indicates whether the robot is abnormal based on a fluctuation in the first difference in chronological order.

11. The robot motion planning device of claim 1 , wherein

the processing circuitry is further configured to display information on one or more differences on a screen of a display device, the one or more differences corresponding to one or more pieces of observation information that are received, respectively.

12. The robot motion planning device of claim 11 , wherein

the processing circuitry is further configured to display a statistic of the one or more differences of each type of object on the screen as the information on the one or more differences.

13. The robot motion planning device of claim 1 , wherein

the processing circuitry is configured to:

acquire a first action-value function corresponding to the first observation information from the action-value function; and

determine the first target position, using the first action-value function.

14. The robot motion planning device of claim 13 , wherein

the processing circuitry is configured to determine a position where the first action-value function is greatest as the first target position.

15. The robot motion planning device of claim 1 , further comprising:

a transmitter configured to transmit a control signal based on the motion plan to the robot.

16. The robot motion planning device of claim 15 , wherein

the processing circuitry is further configured to:

receive first measurement information obtained by measuring a state of the robot while the robot is making a motion based on the control signal; and

update the first difference, using the first measurement information.

17. A robotic system comprising:

a robot; and

a robot motion planning device comprising:

a memory; and

processing circuitry configured to:

receive observation information obtained by observing at least part of a movable range of a robot;

determine, in a case where first observation information is received, a first target position to which the robot is to make a motion, using an action-value function and the first observation information, the action-value function being a function of observation information and a position and being indicative of a value of a motion to be made by the robot;

receive measurement information obtained by measuring a state of the robot;

calculate a first difference corresponding to the first observation information, using the measurement information; and

determine a motion plan of a force-controlled motion of the robot, based on the first target position and the first difference,

wherein the processing circuitry is configured to make a distance between a start point and an end point of the force-controlled motion longer as the first difference is greater.

18. A method comprising:

receiving observation information obtained by observing at least part of a movable range of a robot;

in a case where first observation information is received, determining a first target position to which the robot is to make a motion, using an action-value function and the first observation information, the action-value function being a function of observation information and a position and being indicative of a value of a motion to be made by the robot;

receiving measurement information obtained by measuring a state of the robot;

calculating a first difference corresponding to the first observation information, using the measurement information; and

determining a motion plan of a force-controlled motion of the robot, based on the first target position and the first difference,

wherein the determining the motion plan comprises making a distance between a start point and an end point of the force-controlled motion longer as the first difference is greater.