Operation control method, operation control device, and storage medium

1 . An operation control method comprising:

acquiring sensor data generated when an operator remotely operates an end effector that is configured to grasp or operate an object and in which a plurality of fingers are disposed;

determining a role of each of the plurality of fingers on a basis of the sensor data; and

generating a command for operating the end effector on a basis of the role of each of the plurality of fingers and the sensor data.

2 . The operation control method according to claim 1 , wherein the roles include:

a first role of grasping the object using a target finger together with other fingers;

a second role of operating the object with a balance of forces of the target finger and the other fingers maintained;

a third role of operating the object by applying a force to the object, which is grasped with a balance of forces maintained using the other fingers, using the target finger; and

a fourth role of not applying a force to the object using the target finger.

3 . The operation control method according to claim 2 ,

wherein a value representing a joint angle of each finger of a hand of the operator and a value representing presence/absence of contact of the object with the end effector are included in the sensor data, and

wherein, in a case in which the joint angle of a first finger of interest among fingers of the hand of the operator is equal to or larger than a first threshold used for comparing the joint angle of the first finger of interest, and the object is not brought into contact with the end effector, the role of a finger of the end effector corresponding to the first finger of a hand of the operator is determined to be the fourth role, and in a case in which the joint angle of the first finger is smaller than the first threshold used for comparing the joint angle of the first finger of interest, and the object is brought into contact with the end effector, the role of the finger of the end effector corresponding to the first finger of the hand of the operator is determined to be the first role, the second role, or the third role.

4 . The operation control method according to claim 3 ,

wherein, in a case in which a change of the joint angle of the first finger is smaller than a second threshold used for comparing the change of the joint angle of the first finger, the role of the finger of the end effector that corresponds to the first finger of the hand of the operator is determined to be the first role,

wherein, in a case in which a change of the joint angle of the first finger is equal to or larger than the second threshold used for comparing the change of the joint angle of the first finger, and a relative change between the joint angle of a second finger different from the first finger and the joint angle of the first finger is smaller than a third threshold used for comparing the relative change between the joint angle of the second finger and the joint angle of the first finger, the role of the finger of the end effector that corresponds to the first finger of the hand of the operator is determined to be the second role, and

wherein, in a case in which a change of the joint angle of the first finger is equal to or larger than the second threshold used for comparing the change of the joint angle of the first finger, and a relative change between the joint angle of the second finger and the joint angle of the first finger is equal to or larger than the third threshold used for comparing the relative change between the joint angle of the second finger and the joint angle of the first finger, the role of the finger of the end effector that corresponds to the first finger of the hand of the operator is determined to be the third role.

5 . The operation control method according to claim 1 , wherein the role of each of the plurality of fingers is determined from the sensor data using a machine learning model.

6 . The operation control method according to claim 1 ,

wherein a role of each of the of the plurality of fingers is re-determined from the role of each of the plurality of fingers and the sensor data using a machine learning model, and

wherein the command is generated on the basis of the role of each of the plurality of fingers that has been re-determined.

7 . An operation control device comprising:

a central processing unit; and

a memory that stores executable instructions that, when executed by the central processing unit, facilitate performance of operations, comprising:

acquiring sensor data generated when an operator remotely operates an end effector that is configured to grasp or operate an object and in which a plurality of fingers are disposed;

determining a role of each of the plurality of fingers on a basis of the sensor data; and

generating a command for operating the end effector on a basis of the role of each of the plurality of fingers and the sensor data.

8 . A non-transitory storage medium storing commands, which are computer-readable commands to be executed by a computer, comprising:

acquiring sensor data generated when an operator remotely operates an end effector that is configured to grasp or operate an object and in which a plurality of fingers are disposed;

determining a role of each of the plurality of fingers on a basis of the sensor data; and

generating a command for operating the end effector on a basis of the role of each of the plurality of fingers and the sensor data.