Method for controlling a robot and robot controller
A method for controlling a robot. The method includes providing demonstrations for performing each of a plurality of skills; training from the demonstrations, a robot trajectory model for each skill, each trajectory model is a hidden semi-Markov model having one or more initial states and one or more final states; training, from the demonstrations, a precondition model for each skill comprising, for each initial state, a probability distribution of robot configurations before executing the skill, and a final condition model for each skill comprising, for each final state, a probability distribution of robot configurations after executing the skill; receiving a description of a task, the task includes performing the skills of the plurality of skills in sequence and/or branches; generating a composed robot trajectory model; and controlling the robot according to the composed robot trajectory model to execute the task.
1. A method for controlling a robot, comprising:
providing demonstrations for performing each of a plurality of skills;
training, from the demonstrations, a robot trajectory model for each skill of the skills, wherein each trajectory model is a hidden semi-Markov model having one or more initial states and one or more final states;
training, from the demonstrations, (i) a precondition model for each skill including, for each initial state of the robot trajectory model of the skill, a probability distribution of robot configurations before executing the skill, and (ii) a final condition model for each skill including, for each final state of the robot trajectory model of the skill, a probability distribution of robot configurations after executing the skill;
receiving a description of a task, wherein the task includes performing skills of the plurality of skills in sequence and/or branches;
generating a composed robot trajectory model by:
when two skills are to be performed in sequence in the task, cascading the robot trajectory models of the skills by:
including the states of the trajectory models of the two skills in the composed robot trajectory model, and
calculating a transition probability between each final state of the trajectory model of a first skill of the two skills and each initial state of the trajectory model of a second skill of the two skills as a function of a similarity between the probability distribution of the final condition model of the first skill for the final state of the first skill and the probability distribution of the initial model of the second skill for the initial state for the second skill,
when two skills are performed in branches in the task, combining the robot trajectory models of the skills by:
including the states of the trajectory models of the two skills in the composed robot trajectory model, and
setting a transition probability between states of a first skill of the two skills and states of a second skill of the two skills to zero; and
controlling the robot in accordance with the composed robot trajectory model to execute the task.
2. The method of claim 1 , wherein each skill of the skills includes a manipulation of one or more objects by the robot and the hidden semi-Markov model of each skill is a task-parameterized hidden semi-Markov models including task parameters corresponding to a robot and object configuration in which the skill is applied.
3. The method of claim 2 , further comprising:
training, for each skill of the skills, an effect model including, for each final state of the robot trajectory model of the skill, a probability distribution of how the robot and/or object configuration is changed after executing the skill relative to an initial robot and/or object configuration to which the skill is applied, wherein, when two skills are to be performed in sequence in the task, cascading the robot trajectory models of the two skills including transforming the task-parameterized hidden semi-Markov model of a second skill of the two skills such that its task parameters are task parameters corresponding to the robot and/or object configuration given by the effect model of a first skill of the two skills.
4. The method of claim 3 , further comprising:
affine-transforming the effect model of the second skill such that its task parameters are the task parameters corresponding to the robot and/or object configuration given by the effect model of the first skill.
5. The method of claim 1 , further comprising, when two skills of the skills are to be performed in sequence in the task, using the precondition model of a first skill of the two skills as precondition model for the cascaded robot trajectory models of the skills of the two skills and using the final condition model of a second skill of the two skills as final condition model for the cascaded robot trajectory models of the skills.
6. The method of claim 1 , further comprising:
when two skills of the skills are to be performed in branch in the task, including both the precondition model for a first skill of the two skills and the precondition model for the second skill of the two skills in the precondition model of the combined robot trajectory models of the skills and including both the final condition model for the first skill and the final condition model for the second skill in the precondition model of the combined robot trajectory models of the skills.
7. The method of claim 1 , wherein the similarity between the probability distribution of the final condition model of the first skill for the final state of the first skill and the probability distribution of the initial model of the second skill for the initial state for the second skill is a KL-divergence of the probability distribution of the final condition model of the first skill for the final state of the first skill and the probability distribution of the initial model of the second skill for the initial state for the second skill.
8. The method of claim 1 , wherein the precondition model and the final condition model are task-parameterized Gaussian mixture models.
9. The method of claim 1 , wherein generating the composed robot trajectory model includes repeatedly cascading robot trajectory models, cascaded robot trajectory models, and combined robot trajectory models, and combining robot trajectory models, cascaded robot trajectory models, and combined robot trajectory in accordance with the task such that the robot trajectory model is a trajectory model for a whole task.
10. A robot controller configured to control a robot, the robot controller configured to:
provide demonstrations for performing each of a plurality of skills;
train, from the demonstrations, a robot trajectory model for each skill of the skills, wherein each trajectory model is a hidden semi-Markov model having one or more initial states and one or more final states;
train, from the demonstrations, (i) a precondition model for each skill including, for each initial state of the robot trajectory model of the skill, a probability distribution of robot configurations before executing the skill, and (ii) a final condition model for each skill including, for each final state of the robot trajectory model of the skill, a probability distribution of robot configurations after executing the skill;
receive a description of a task, wherein the task includes performing skills of the plurality of skills in sequence and/or branches;
generate a composed robot trajectory model by:
when two skills are to be performed in sequence in the task, cascading the robot trajectory models of the skills by:
including the states of the trajectory models of the two skills in the composed robot trajectory model, and
calculating a transition probability between each final state of the trajectory model of a first skill of the two skills and each initial state of the trajectory model of a second skill of the two skills as a function of a similarity between the probability distribution of the final condition model of the first skill for the final state of the first skill and the probability distribution of the initial model of the second skill for the initial state for the second skill,
when two skills are performed in branches in the task, combining the robot trajectory models of the skills by:
including the states of the trajectory models of the two skills in the composed robot trajectory model, and
setting a transition probability between states of a first skill of the two skills and states of a second skill of the two skills to zero; and
control the robot in accordance with the composed robot trajectory model to execute the task.
11. A non-transitory computer readable medium on which are stored instructions for controlling a robot, the instructions, when executed by a processor, causing the processor to perform the following:
providing demonstrations for performing each of a plurality of skills;
training, from the demonstrations, a robot trajectory model for each skill of the skills, wherein each trajectory model is a hidden semi-Markov model having one or more initial states and one or more final states;
training, from the demonstrations, (i) a precondition model for each skill including, for each initial state of the robot trajectory model of the skill, a probability distribution of robot configurations before executing the skill, and (ii) a final condition model for each skill including, for each final state of the robot trajectory model of the skill, a probability distribution of robot configurations after executing the skill;
receiving a description of a task, wherein the task includes performing skills of the plurality of skills in sequence and/or branches;
generating a composed robot trajectory model by:
when two skills are to be performed in sequence in the task, cascading the robot trajectory models of the skills by:
including the states of the trajectory models of the two skills in the composed robot trajectory model, and
calculating a transition probability between each final state of the trajectory model of a first skill of the two skills and each initial state of the trajectory model of a second skill of the two skills as a function of a similarity between the probability distribution of the final condition model of the first skill for the final state of the first skill and the probability distribution of the initial model of the second skill for the initial state for the second skill,
when two skills are performed in branches in the task, combining the robot trajectory models of the skills by:
including the states of the trajectory models of the two skills in the composed robot trajectory model, and
setting a transition probability between states of a first skill of the two skills and states of a second skill of the two skills to zero; and
controlling the robot in accordance with the composed robot trajectory model to execute the task.