Robotic demonstration retrieval systems and methods
A robot system includes a selection module configured to select a stored demonstration for a robot from a database of stored demonstrations for different tasks of the robot; an encoder module of an attention model, the encoder module configured to determine a similarity value reflecting a similarity between: a user input demonstration for the robot; and the stored demonstration for the robot; and an indicator module configured to indicate whether the stored demonstration is the same as the user input demonstration and belongs to the same task based on the similarity value.
1 . A robot system comprising:
one or more processors; and
memory including code that, when executed by the one or more processors, performs to:
select a stored demonstration for a robot from a database of stored demonstrations for different tasks of the robot;
using an encoder of an attention model, determine a similarity value reflecting a similarity between:
a user input demonstration for the robot; and
the selected one of the stored demonstrations for the robot;
wherein the similarity value is determined based on masking one or more scaled outputs and generating a concatenation based on the user input demonstration and the selected one of the stored demonstrations;
indicate whether the selected one of the stored demonstrations corresponds to the user input demonstration and belongs to the same task based on the similarity value; and
actuate actuators of the robot based on a policy trained using the stored demonstration output for augmenting training of the policy,
wherein the encoder is trained using a training dataset of training demonstrations and a listwise loss over concatenated pairs of training demonstrations, wherein each of the concatenated pairs of training demonstrations are based on a task identifier associated with each training demonstration,
the training demonstrations each including a temporal sequence of observation-action tuples of successive behaviors for tasks.
2 . The robot system of claim 1 wherein the code, when executed by the one or more processors, performs to indicate that the selected one of the stored demonstrations corresponds to the user input demonstration and belongs to the same task when the similarity value is greater than a predetermined value.
3 . The robot system of claim 2 wherein the code, when executed by the one or more processors, performs to output the selected one of the stored demonstrations for augmenting training of the policy for the task in response to a determination that the user input demonstration belongs to the same task as the selected one of the stored demonstration.
4 . The robot system of claim 2 wherein the code, when executed by the one or more processors, performs to indicate that the selected one of the stored demonstrations does not correspond to the user input demonstration when the similarity value is less than the predetermined value.
5 . The robot system of claim 1 wherein:
the selected one of the stored demonstration includes a first time series of at least 6 degree of freedom (DoF) poses of an end effector of the robot; and
the user input demonstration includes a second time series of at least 6 DoF poses of an end effector of the robot.
6 . The robot system of claim 1 wherein the encoder includes a stack of 6 identical layers.
7 . The robot system of claim 6 wherein each of the six identical layers includes a first sub-layer and a second sub-layer.
8 . The robot system of claim 7 wherein the first sublayers perform multi-head attention.
9 . The robot system of claim 8 wherein the multi-head attention includes performing scaled dot product attention.
10 . The robot system of claim 7 wherein the second sublayers define a position wise fully connected feed forward network.
11 . The robot system of claim 7 wherein the six identical layers include residual connections around the first and second sub-layers.
12 . The robot system of claim 1 wherein the code, when executed by the one or more processors, performs to:
select a second one of the stored demonstrations for the robot from the database of stored demonstrations in response to a determination that the selected one of the stored demonstrations does not correspond to the user input demonstration;
determine a second similarity value reflecting a second similarity between:
the user input demonstration for the robot; and
the second one of the stored demonstrations for the robot; and
indicate whether the second one of the stored demonstrations corresponds to the user input demonstration and belongs to the same task based on the second similarity value.
13 . The robot system of claim 1 wherein the encoder has a Transformer architecture.
14 . The robot system of claim 1 wherein the code, when executed by the one or more processors, performs to determine one or more other ones of the stored demonstrations associated with the selected one of the stored demonstrations in response to the determination that the one or more other stored demonstrations corresponds to the user input demonstration and belong to the same task.
15 . The robot system of claim 14 wherein the code, when executed by the one or more processors, further performs to actuate actuators of the robot based on the policy trained using the one or more other ones of the stored demonstrations in response to a determination that the one or more other ones of the stored demonstrations are the same as the user input demonstration and belong to the same task.
16 . A method for operating a robot system, the method comprising:
selecting a stored demonstration for a robot from a database of stored demonstrations for different tasks of the robot;
determining, using an attention model, a similarity value reflecting a similarity between:
a user input demonstration for the robot; and
one or more of the stored demonstrations for the robot;
wherein the determining the similarity value includes determining the similarity value based on masking one or more scaled outputs and generating a concatenation based on the user input demonstration and the selected one of the stored demonstrations;
producing an output indicating whether the one or more stored demonstrations correspond to as the user input demonstration and belong to the same task based on the similarity value; and
actuating actuators of the robot based on a policy trained using the selected one of the stored demonstrations output for augmenting training of the policy,
wherein the attention model is trained using a training dataset of training demonstrations and a listwise loss over concatenated pairs of training demonstrations, wherein each of the concatenated pairs of training demonstrations are based on a task identifier associated with each training demonstration,
the training demonstrations each including a temporal sequence of observation-action tuples of successive behaviors for tasks.
17 . The method of claim 16 wherein the producing the output further includes producing an output that indicates that the one or more stored demonstrations correspond to the user input demonstration and belong to the same task when their similarity value is greater than a predetermined value.
18 . The method of claim 17 further comprising outputting the one or more stored demonstrations for augmenting training of the policy for the task in response to a determination that the user input demonstration belongs to the same task as the selected one of the stored demonstrations.
19 . The method of claim 17 wherein the producing the output further includes producing an output that indicates that the one or more stored demonstrations do not correspond to the user input demonstration when their similarity value is less than the predetermined value.
20 . The method of claim 16 further comprising training the attention model using the training dataset of training demonstrations and a listwise loss.
21 . The method of claim 19 further comprising:
determining one or more other stored demonstrations associated with the one or more stored demonstrations in response to the determination that the one or more stored demonstrations correspond to the user input demonstration and belongs to the same task; and
actuating the actuators of the robot further based on the one or more other stored demonstrations.
22 . The method of claim 19 further wherein:
the one or more stored demonstrations include a first time series of at least 6 degree of freedom (DoF) poses of an end effector of the robot; and
the user input demonstration includes a second time series of at least 6 DoF poses of an end effector of the robot.
23 . A method for operating a robot system comprising:
obtaining a trained policy stored in a memory; and
actuating actuators of a robot using the trained policy thereby performing a task defined by one or more user input demonstrations; and
wherein the trained policy includes a trained attention model that is trained using a training dataset of training demonstrations and a listwise loss over concatenated pairs of the training demonstrations, wherein each of the concatenated pairs of training demonstrations are based on a task identifier associated with each training demonstration,
the training demonstrations each including a temporal sequence of observation-action tuples of successive behaviors for tasks, and
wherein the training dataset of demonstrations, which includes the one or more user input demonstrations for the robot, and which is used to train the trained policy for actuating the actuators of the robot, is augmented with one or more stored demonstrations for the robot when the stored demonstrations are determined to correspond to the one or more user input demonstration and belong to the same task when their similarity values are greater than a predetermined value,
the augmenting including masking one or more scaled outputs and generating a concatenation based on the one or more user input demonstrations and the one or more stored demonstrations.
24 . A robot system comprising:
a first circuit configured to select a stored demonstration for a robot from a database of stored demonstrations for different tasks of the robot;
a second circuit configured to determine, using an attention model, a similarity value reflecting a similarity between:
a user input demonstration for the robot; and
the selected one of the stored demonstrations for the robot;
wherein the determining the similarity value includes determining the similarity value based on masking one or more scaled outputs and generating a concatenation based on the user input demonstration and the selected one of the stored demonstrations;
a third circuit configured to indicate whether the selected one of the stored demonstrations corresponds to the user input demonstration and belongs to the same task based on the similarity value; and
a fourth circuit configured to actuate actuators of the robot based on a policy trained using the selected one of the stored demonstrations output for augmenting training of the policy,
wherein the attention model is trained using a training dataset of training demonstrations and a listwise loss over concatenated pairs of training demonstrations, wherein each of the concatenated pairs of training demonstrations are based on a task identifier associated with each training demonstration,
the training demonstrations each including a temporal sequence of observation-action tuples of successive behaviors for tasks.