Robot remote operation control device, robot remote operation control system, robot remote operation control method, and program
A robot remote operation control device includes, in robot remote operation control for an operator to remotely operate a robot capable of gripping an object, an information acquisition unit that acquires operator state information on a state of the operator who operates the robot, an intention estimation unit that estimates a motion intention of the operator who causes the robot to perform a motion, on the basis of the operator state information, and a gripping method determination unit that determines a gripping method for the object on the basis of the estimated motion intention of the operator.
1 . A robot remote operation control device comprising:
at least one memory that stores executable instructions; and
at least one processor configured to process the executable instructions that, when executed by the at least one processor, facilitate performance of operations that enable an operator to remotely operate a robot capable of gripping an object, the operations comprising:
acquiring operator state information on a state of the operator who operates the robot;
estimating a motion intention of the operator who causes the robot to perform a motion, on the basis of the operator state information, resulting in an estimated determined motion intention;
determining a gripping method for the object on the basis of the estimated motion intention of the operator, resulting in a determined gripping method,
wherein the estimating of the motion intention comprises:
classifying a gripping posture of the operator into a predetermined classification of gripping postures from a handshape of the operator,
estimating the motion intention of the operator on a basis of a classification result of the gripping posture, and
on the basis of the operator state information and robot state information representing a state of the robot, estimating a future trajectory of the operator's hand tip which the operator intends as the motion intention, and
wherein the determining of the gripping method comprises:
calculating contact points of fingers of the robot to the object on the basis of the classification result of the gripping posture, a shape of the object, physical parameters of the object, and torques that can be output by the robot,
calculating joint angles of the fingers on the basis of the contact points, and
correcting motions of the fingers of the robot operated by the operator on the basis of the motion intention and the joint angles; and
operating the robot in accordance with the determined motion intention and the determined gripping method.
2 . The robot remote operation control device according to claim 1 , wherein the classifying of the gripping posture comprises classifying the gripping posture on the basis of the operator state information.
3 . The robot remote operation control device according to claim 1 , wherein the estimating of the motion intention comprises:
estimating at least one of:
a way of gripping an object desired to be gripped, or
the object desired to be gripped on the basis of the operator state information.
4 . The robot remote operation control device according to claim 1 , wherein the estimating of the motion intention comprises:
estimating which object is desired to be gripped on the basis of the operator state information, and
estimating a way of gripping the object desired to be gripped, related to the estimated object.
5 . The robot remote operation control device according to claim 4 ,
wherein the estimating of the way of gripping the object comprises estimating the way of gripping the object on the basis of the operator state information, and
wherein the estimating of the object desired to be gripped comprises estimating the object desired to be gripped on the basis of an estimated manner of gripping the object desired to be gripped.
6 . The robot remote operation control device according to claim 1 , wherein the operator state information comprises at least one of:
line-of-sight information of the operator,
movement information of the operator's arm, or
movement information of the operator's head.
7 . The robot remote operation control device according to claim 4 , wherein the operations further comprise:
acquiring position information of the object, resulting in acquired position information, wherein the estimating of which object is desired to be gripped is based further on the acquired position information.
8 . The robot remote operation control device according to claim 1 , wherein the determining of the gripping method further comprises:
acquiring position information of the gripping portion provided in the robot, and
correcting the position information of the gripping portion on the basis of the operator state information.
9 . The robot remote operation control device according to claim 8 , further comprising:
creating a robot state image based on:
the information on the object,
the position information of the gripping portion,
the operator state information, and
the corrected position information of the gripping portion, wherein the estimating of the motion intention is based further on the robot state image.
10 . A robot remote operation control system comprising:
a robot comprising:
a gripping portion that grips the object;
the robot remote operation control device according to claim 1 ;
an environment sensor that detects position information of the object and position information of the gripping portion; and
a sensor that detects operator state information on the state of the operator who operates the robot, resulting in a determined state of the operator, wherein the estimating of the motion intention comprises:
classifying a gripping posture of the operator into a predetermined classification of gripping postures from a handshape of the operator,
estimating the motion intention of the operator on a basis of a classification result of the gripping posture, and
on the basis of the operator state information and robot state information representing a state of the robot, estimating a future trajectory of the operator's hand tip which the operator intends as the motion intention, and
wherein the determining of the gripping method comprises:
calculating contact points of fingers of the robot to the object on the basis of the classification result of the gripping posture, a shape of the object, physical parameters of the object, and torques that can be output by the robot,
calculating joint angles of the fingers on the basis of the contact points, and
correcting motions of the fingers of the robot operated by the operator on the basis of the motion intention and the joint angles; and
operating the robot in accordance with the determined state of the operator.
11 . A robot remote operation control method comprising:
acquiring, by a computing system comprising at least one processor, operator state information on a state of an operator who operates a robot;
estimating, by the computing system, a motion intention of the operator, on a basis of the operator state information, resulting in an estimated motion intention;
determining, by the computing system, a gripping method for the object on a basis of the estimated motion intention, resulting in a determined gripping method,
wherein the estimating of the motion intention comprises:
classifying a gripping posture of the operator into a predetermined classification of gripping postures from a handshape of the operator,
estimating the motion intention of the operator on a basis of a classification result of the gripping posture, and
on the basis of the operator state information and robot state information representing a state of the robot, estimating, by the computing system, a future trajectory of the operator's hand tip which the operator intends as the motion intention, and
wherein the determining of the gripping method comprises:
calculating contact points of fingers of the robot to the object on the basis of the classification result of the gripping posture, a shape of the object, physical parameters of the object, and torques that can be output by the robot,
calculating joint angles of the fingers on the basis of the contact points, and
correcting motions of the fingers of the robot operated by the operator on the basis of the motion intention and the joint angles; and
operating, by the computing system, the robot in accordance with the determined motion intention and the determined gripping method.
12 . A non-transitory machine-readable medium comprising executable instructions that, when executed by at least one processor of computer equipment, facilitate performance of operations to remotely operate a robot capable of gripping an object, the operations comprising:
acquiring operator state information on a state of an operator who operates the robot;
estimating a motion intention of the operator on the basis of the operator state information, resulting in an estimated motion intention;
determining a gripping method for the object on the basis of the estimated motion intention of the operator, resulting in a determined gripping method,
wherein the estimating of the motion intention comprises:
classifying a gripping posture of the operator into a predetermined classification of gripping postures from a handshape of the operator,
estimating the motion intention of the operator on the basis of a classification result of the gripping posture, and
on the basis of the operator state information and a robot state information representing a state of the robot, estimating a future trajectory of the operator's hand tip which the operator intends as the motion intention, and
wherein the determining of the gripping method comprises:
calculating contact points of fingers of the robot to the object on the basis of the classification result of the gripping posture, a shape of the object, physical parameters of the object, and torques that can be output by the robot,
calculating joint angles of the fingers on the basis of the contact points, and
correcting motions of the fingers of the robot operated by the operator on the basis of the motion intention and the joint angles; and
operating the robot in accordance with the determined motion intention and the determined gripping method.