ROBOTS, TELE-OPERATION SYSTEMS, COMPUTER PROGRAM PRODUCTS, AND METHODS OF OPERATING THE SAME
The present disclosure describes robots, tele-operation systems, methods, and computer program products where a robot is selectively operable in a plurality of control modes. Based on identification of a fault condition (when the robot fails to act in a suitable or sufficient manner), a control mode of the robot can be changed to provide a human operator with more explicit control over the robot. In this way, the fault condition can be resolved by human operator input, and the control modes, AI, or control paradigm for the robot can be trained to perform better in the future.
1 . A method of operating a tele-operation system, the tele-operation system comprising at least one processor, an operator input interface which receives input from an operator of the tele-operation system, and a communication interface that communicatively couples the at least one processor to a robot, the robot selectively operable in a plurality of control modes including a first control mode that corresponds to a first level of robot autonomy and at least a second control mode that corresponds to a second level of robot autonomy, the second level of robot autonomy being less autonomous than the first level of robot autonomy, the method comprising:
identifying a fault condition of the robot during operation of the robot in the first control mode;
in response to identifying the fault condition during operation of the robot in the first control mode, processing operator input received by the operator input interface; and
sending at least one instruction based on the operator input to the robot via the communication interface, to operate the robot in the second control mode,
wherein instructions sent to the robot from the tele-operation system to operate the robot in the second control mode are based on more explicit input from the operator than for operation of the robot in the first control mode.
2 . The method of claim 1 , wherein:
the first level of robot autonomy comprises full robot autonomy, and no instructions are sent from the tele-operation system to the robot for operation of the robot in the first control mode; and
the second level of robot autonomy comprises partial robot autonomy, and sending the at least one instruction based on the operator input to the robot comprises: sending at least one action instruction to the robot, the at least one action instruction indicating at least one action to be performed by the robot as received by the operator input interface from the operator of the tele-operation system.
3 . The method of claim 2 , wherein the plurality of control modes further includes a third control mode that corresponds to a third level of robot autonomy, the third level of robot autonomy being less autonomous than the second level of robot autonomy, where instructions sent to the robot from the tele-operation system to operate the robot in the third control mode are based on more explicit input from the operator than instructions sent to the robot from the tele-operation system for operation of the robot in the second control mode, wherein the third level of robot autonomy comprises no robot autonomy, and instructions sent to the robot from the tele-operation system to operate the robot in the third control mode include at least one movement instruction from the operator of the tele-operation system received by the operator input interface, the at least one movement instruction indicating at least one movement to be emulated by at least one actuatable member of the robot.
4 . The method of claim 1 , wherein:
the first level of robot autonomy comprises full robot autonomy, and no instructions are sent from the tele-operation system to the robot for operation of the robot in the first control mode; and
the second level of robot autonomy comprises no robot autonomy, and instructions sent to the robot from the tele-operation system to operate the robot in the second control mode include at least one movement instruction from the operator of the tele-operation system received by the operator input interface, the at least one movement instruction indicating at least one movement to be emulated by at least one actuatable member of the robot.
5 . The method of claim 4 , wherein:
the plurality of control modes further includes a third control mode that corresponds to a third level of robot autonomy, the third level of robot autonomy being less autonomous than the first level of robot autonomy and more autonomous than the second level of robot autonomy;
instructions sent to the robot from the tele-operation system to operate the robot in the third control mode are based on less explicit input from the operator than instructions sent to the robot from the tele-operation system for operation of the robot in the second control mode; and
instructions sent to the robot from the tele-operation system to operate the robot in the third control mode are based on more explicit input from the operator than for operation of the robot in the first control mode, wherein the third level of robot autonomy comprises partial robot autonomy, and instructions sent to the robot from the tele-operation system to operate the robot in the third control mode include at least one action instruction indicating at least one action to be performed by the robot as received by the operator input interface from the operator of the tele-operation system.
6 . The method of claim 1 , wherein:
the first level of robot autonomy comprises partial robot autonomy;
instructions sent to the robot from the tele-operation system to operate the robot in the first control mode include at least one action instruction indicating at least one action to be performed by the robot as received by the operator input interface from the operator of the tele-operation system;
the second level of robot autonomy comprises no robot autonomy; and
sending the at least one instruction based on the operator input to the robot comprises: sending at least one movement instruction to the robot, the at least one movement instruction indicating at least one movement to be emulated by at least one actuatable member of the robot as received by the operator input interface from the operator of the tele-operation system.
7 . The method of claim 6 , wherein:
the plurality of control modes further includes a third control mode that corresponds to a third level of robot autonomy, the third level of robot autonomy being more autonomous than the first level of robot autonomy; and
the third level of robot autonomy comprises full robot autonomy, and no instructions are sent from the tele-operation system to the robot for operation of the robot in the third control mode.
8 . The method of claim 1 , wherein the plurality of control modes further includes a third control mode that corresponds to a third level of robot autonomy, the third level of robot autonomy being less autonomous than the second level of robot autonomy, and the method further comprising:
identifying another fault condition of the robot during operation of the robot in the second control mode;
in response to identifying the another fault condition during operation of the robot in the second control mode, processing further operator input received by the operator input interface; and
sending at least one further instruction based on the further operator input to the robot via the communication interface, to operate the robot in the third control mode.
9 . The method of claim 1 , wherein:
identifying a fault condition of the robot comprises identifying, by the tele-operation system, the fault condition based on fault data received from the robot via the communication interface, the fault data indicating the fault condition of the robot.
10 . The method of claim 1 , further comprising:
training the first control mode based on at least input from the operator of the tele-operation system received by the operator input interface for operating the robot in the second control mode.
11 . A tele-operation system comprising:
at least one processor;
an operator input interface which receives input from an operator of the tele-operation system;
a communication interface that communicatively couples the at least one processor to a robot, the robot selectively operable in a plurality of control modes including a first control mode that corresponds to a first level of robot autonomy and at least a second control mode that corresponds to a second level of robot autonomy, the second level of robot autonomy being less autonomous than the first level of robot autonomy;
at least one non-transitory processor-readable storage medium communicatively coupled to the at least one processor, the at least one non-transitory processor-readable storage medium storing processor-executable instructions which, when executed by the at least one processor, cause the tele-operation system to:
identify a fault condition of the robot during operation of the robot in the first control mode;
in response to identifying the fault condition during operation of the robot in the first control mode, process operator input received by the operator input interface; and
send at least one instruction based on the operator input to the robot via the communication interface, to operate the robot in the second control mode,
wherein instructions sent to the robot from the tele-operation system to operate the robot in the second control mode are based on more explicit input from the operator than for operation of the robot in the first control mode.
12 . The tele-operation system of claim 11 , wherein:
the first level of robot autonomy comprises full robot autonomy, and no instructions are sent from the tele-operation system to the robot for operation of the robot in the first control mode; and
the second level of robot autonomy comprises partial robot autonomy, and the processor-executable instructions which cause the robot to send at least one instruction based on the operator input to the robot via the communication interface cause the robot to: send at least one action instruction to the robot, the at least one action instruction indicating at least one action to be performed by the robot as received by the operator input interface from the operator of the tele-operation system.
13 . The tele-operation system of claim 12 , wherein the plurality of control modes further includes a third control mode that corresponds to a third level of robot autonomy, the third level of robot autonomy being less autonomous than the second level of robot autonomy, where instructions sent to the robot from the tele-operation system to operate the robot in the third control mode are based on more explicit input from the operator than instructions sent to the robot from the tele-operation system for operation of the robot in the second control mode, wherein the third level of robot autonomy comprises no robot autonomy, and instructions sent to the robot from the tele-operation system to operate the robot in the third control mode include at least one movement instruction from the operator of the tele-operation system received by the operator input interface, the at least one movement instruction indicating at least one movement to be emulated by at least one actuatable member of the robot.
14 . The tele-operation system of claim 11 , wherein the plurality of control modes further includes a third control mode that corresponds to a third level of robot autonomy, the third level of robot autonomy being less autonomous than the second level of robot autonomy, and the processor-executable instructions further cause the tele-operation system to:
identify another fault condition of the robot during operation of the robot in the second control mode;
in response to identifying the another fault condition during operation of the robot in the second control mode, process further operator input received by the operator input interface; and
send at least one further instruction based on the further operator input to the robot via the communication interface, to operate the robot in the third control mode.
15 . The tele-operation system of claim 11 , wherein the processor-executable instructions further cause the tele-operation system to:
train the first control mode based on at least input from the operator of the tele-operation system received by the operator input interface for operating the robot in the second control mode.
16 . The tele-operation system of claim 11 , wherein the operator input interface comprises an action input interface where the operator selects an action to be performed by the robot, the action input interface including a graphical user interface or a point-and-click interface.
17 . The tele-operation system of claim 11 , wherein the operator input interface comprises a plurality of sensors wearable by the operator, the plurality of sensors configured to capture movement by the operator as at least one movement instruction, for emulation of the movement by the robot.
18 . A computer program product comprising processor-executable instructions or data that, when the computer program product is stored in a non-transitory processor-readable storage medium of a tele-operation system, and the computer program product is executed by at least one processor of the tele-operation system, the at least one processor communicatively coupled to the non-transitory processor-readable storage medium, causes the tele-operation system to:
identify a fault condition of a robot, the robot communicatively coupled to the tele-operation system by a communication interface and the robot selectively operable in a plurality of control modes including a first control mode that corresponds to a first level of robot autonomy and at least a second control mode that corresponds to a second level of robot autonomy, the second level of robot autonomy being less autonomous than the first level of robot autonomy, where the fault condition occurs during operation of the robot in the first control mode;
in response to identifying the fault condition, process operator input received from an operator of the tele-operation system by an operator input interface; and
send at least one instruction based on the operator input to the robot via the communication interface, to operate the robot in the second control mode,
wherein instructions sent to the robot from the tele-operation system to operate the robot in the second control mode are based on more explicit input from the operator than for operation of the robot in the first control mode.
19 . The computer program product of claim 18 , wherein:
the first level of robot autonomy comprises full robot autonomy, and no instructions are sent from the tele-operation system to the robot for operation of the robot in the first control mode; and
the second level of robot autonomy comprises partial robot autonomy, and the processor-executable instructions or data of the computer program product which cause the robot to send at least one instruction based on the operator input to the robot via the communication interface cause the robot to: send at least one action instruction to the robot, the at least one action instruction indicating at least one action to be performed by the robot as received by the operator input interface from the operator of the tele-operation system.
20 . The computer program product of claim 19 , wherein the plurality of control modes further includes a third control mode that corresponds to a third level of robot autonomy, the third level of robot autonomy being less autonomous than the second level of robot autonomy, where instructions sent to the robot from the tele-operation system to operate the robot in the third control mode are based on more explicit input from the operator than instructions sent to the robot from the tele-operation system for operation of the robot in the second control mode, wherein the third level of robot autonomy comprises no robot autonomy, and instructions sent to the robot from the tele-operation system to operate the robot in the third control mode include at least one movement instruction from the operator of the tele-operation system received by the operator input interface, the at least one movement instruction indicating at least one movement to be emulated by at least one actuatable member of the robot.
21 . The computer program product of claim 18 , wherein the plurality of control modes further includes a third control mode that corresponds to a third level of robot autonomy, the third level of robot autonomy being less autonomous than the second level of robot autonomy, and the processor-executable instructions or data of the computer program product further cause the tele-operation system to:
identify another fault condition of the robot during operation of the robot in the second control mode;
in response to identifying the another fault condition during operation of the robot in the second control mode, process further operator input received by the operator input interface; and
send at least one further instruction based on the further operator input to the robot via the communication interface, to operate the robot in the third control mode.