Skill templates for robotic demonstration learning

1. A method performed by one or more computers, the method comprising:

receiving a skill template for a task to be performed by a robot, wherein the skill template defines a state machine having a plurality of subtasks and one or more respective transition conditions between one or more of the subtasks,

wherein the skill template indicates which of the subtasks are demonstration subtasks to be refined using local demonstration data;

obtaining a base control policy for a demonstration subtask of the skill template;

receiving, for the demonstration subtask of the skill template, local demonstration data generated from a user demonstrating with the robot how to perform the demonstration subtask, wherein the local demonstration data comprises a set of task state representations generated from a plurality of sensor streams while the user demonstrates the demonstration subtask using the robot;

refining the base control policy for the demonstration subtask using the local demonstration data, wherein a refined based control policy for the demonstration subtask is configured to generate, using one or more input sensor streams, a command to be executed by the robot; and

executing the skill template on the robot, thereby causing the robot to transition through the state machine defined by the skill template to perform the task, including performing commands generated by the refined base control policy.

2. The method of claim 1 , wherein receiving local demonstration data comprises receiving local demonstration data for each of a plurality of transition conditions for the demonstration subtask of the skill template.

3. The method of claim 1 , wherein the skill template specifies that fewer than all transition conditions of the demonstration subtask require local demonstration data.

4. The method of claim 1 , wherein the skill template is hardware-agnostic.

5. The method of claim 4 , further comprising adapting the skill template for a plurality of different robot models including generating respective local demonstration data by the plurality of different robot models.

6. The method of claim 1 , wherein the skill template specifies, for each subtask of the plurality of subtasks, which input sensor streams are required to complete the subtask.

7. The method of claim 6 , wherein the base control policy was generated from demonstrations on a plurality of different robot models.

8. The method of claim 1 , wherein a first subtask of the skill template references a different second skill template having a plurality of second subtasks.

9. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

receiving a skill template for a task to be performed by a robot, wherein the skill template defines a state machine having a plurality of subtasks and one or more respective transition conditions between one or more of the subtasks,

wherein the skill template indicates which of the subtasks are demonstration subtasks to be refined using local demonstration data;

obtaining a base control policy for a demonstration subtask of the skill template;

receiving, for the demonstration subtask of the skill template, local demonstration data generated from a user demonstrating with the robot how to perform the demonstration subtask, wherein the local demonstration data comprises a set of task state representations generated from a plurality of sensor streams while the user demonstrates the demonstration subtask using the robot;

refining the base control policy for the demonstration subtask using the local demonstration data, wherein a refined based control policy for the demonstration subtask is configured to generate, using one or more input sensor streams, a command to be executed by the robot; and

executing the skill template on the robot, thereby causing the robot to transition through the state machine defined by the skill template to perform the task, including performing commands generated by the refined base control policy.

10. The system of claim 9 , wherein receiving local demonstration data comprises receiving local demonstration data for each of a plurality of transition conditions for the demonstration subtask of the skill template.

11. The system of claim 9 , wherein the skill template specifies that fewer than all transition conditions of the demonstration subtask require local demonstration data.

12. The system of claim 9 , wherein the skill template is hardware-agnostic.

13. The system of claim 12 , wherein the operations further comprise adapting the skill template for a plurality of different robot models including generating respective local demonstration data by the plurality of different robot models.

14. The system of claim 9 , wherein the skill template specifies, for each subtask of the plurality of subtasks, which input sensor streams are required to complete the subtask.

15. The system of claim 14 , wherein the base control policy was generated from demonstrations on a plurality of different robot models.

16. The system of claim 9 , wherein a first subtask of the skill template references a different second skill template having a plurality of second subtasks.

17. One or more non-transitory computer storage media encoded with computer program instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

receiving a skill template for a task to be performed by a robot, wherein the skill template defines a state machine having a plurality of subtasks and one or more respective transition conditions between one or more of the subtasks,

wherein the skill template indicates which of the subtasks are demonstration subtasks to be refined using local demonstration data;

obtaining a base control policy for a demonstration subtask of the skill template;

receiving, for the demonstration subtask of the skill template, local demonstration data generated from a user demonstrating with the robot how to perform the demonstration subtask, wherein the local demonstration data comprises a set of task state representations generated from a plurality of sensor streams while the user demonstrates the demonstration subtask using the robot;

refining the base control policy for the demonstration subtask using the local demonstration data, wherein a refined based control policy for the demonstration subtask is configured to generate, using one or more input sensor streams, a command to be executed by the robot; and

executing the skill template on the robot, thereby causing the robot to transition through the state machine defined by the skill template to perform the task, including performing commands generated by the refined base control policy.

18. The one or more non-transitory computer storage media of claim 17 , wherein receiving local demonstration data comprises receiving local demonstration data for each of a plurality of transition conditions for the demonstration subtask of the skill template.

19. The one or more non-transitory computer storage media of claim 17 , wherein the skill template specifies that fewer than all transition conditions of the demonstration subtask require local demonstration data.

20. The one or more non-transitory computer storage media of claim 17 , wherein the skill template is hardware-agnostic.