Systems and methods for learning and generating movement policies for a dynamical system

1. A computer-implemented method for generating a trajectory of a dynamical system, comprising:

generating, using a first recurrent mixture density network (RMDN), a desired effect on the dynamical system in response to a plurality of system states and effects, wherein the first RMDN outputs the desired effect and a confidence value;

predicting, using a second RMDN, a respective corresponding effect on the dynamical system in response to each of a plurality of possible system actions, wherein the second RMDN outputs each respective corresponding effect and respective confidence value;

comparing each respective corresponding effect on the dynamical system to the desired effect on the dynamical system; and

selecting a next system action in the trajectory of the dynamical system based on the comparison.

2. The computer-implemented method of claim 1 , further comprising generating the possible system actions.

3. The computer-implemented method of claim 1 , wherein the possible system actions are user-defined.

4. The computer-implemented method of claim 2 , wherein the possible system actions are generated using a recurrent neural network (RNN).

5. The computer-implemented method of claim 4 , wherein the RNN comprises a plurality of long short-term memory (LSTM) units.

6. The computer-implemented method of claim 5 , wherein at least one of the LSTM units is a peephole LSTM unit.

7. The computer-implemented method of claim 5 , wherein the RNN includes 1 layer and 16 LSTM units.

8. The computer-implemented method of claim 1 , wherein the first RMDN is trained with demonstration data.

9. The computer-implemented method of claim 1 , wherein the second RMDN is trained with demonstration data.

10. The computer-implemented method of claim 1 , wherein the trajectory comprises a plurality of sequential system states over time.

11. The computer-implemented method of claim 1 , wherein the dynamical system is a robotic system.

12. The computer-implemented method of claim 11 , wherein the robotic system is configured to execute a pouring task.

13. The computer-implemented method of claim 12 , wherein the plurality of system states include at least one of a characteristic of a container, an initial volume in the container, a current volume in the container, a target pour volume, an amount poured, a rotation angle of the container, or an angular velocity of a motor.

14. The computer-implemented method of claim 13 , wherein the next system action is the angular velocity of the motor.

15. A system for generating a trajectory of a dynamical system, comprising:

a first recurrent mixture density network (RMDN) configured to generate a desired effect on the dynamical system in response to a plurality of system states and effects, wherein the first RMDN is configured to output the desired effect and a confidence value;

a second RMDN configured to predict a respective corresponding effect on the dynamical system in response to each of a plurality of possible system actions, wherein the second RMDN is configured to output each respective corresponding effect and respective confidence value; and

a processor and a memory operably connected to the processor, wherein the memory has computer-executable instructions stored thereon that, when executed by the processor, cause the processor to:

compare each respective corresponding effect on the dynamical system to the desired effect on the dynamical system; and

select a next system action in the trajectory of the dynamical system based on the comparison.

16. The system of claim 15 , further comprising an action generator configured to generate the possible system actions.

17. The system of claim 15 , further comprising a recurrent neural network (RNN) configured to generate the possible system actions.

18. The system of claim 17 , wherein the RNN comprises a plurality of long short-term memory (LSTM) units.

19. The system of claim 15 , further comprising a robot, wherein the memory has further computer-executable instructions stored thereon that, when executed by the processor, cause the processor to transmit a command including the next system action to the robot.

20. The system of claim 19 , wherein the robot is configured to execute a pouring task.

21. The system of claim 20 , wherein the plurality of system states include at least one of a characteristic of a container, an initial volume in the container, a current volume in the container, a target pour volume, an amount poured, a rotation angle of the container, or an angular velocity of a motor.

22. The system of claim 21 , wherein the next system action is the angular velocity of the motor.

23. The computer-implemented method of claim 1 , wherein the step of comparing each respective corresponding effect on the dynamical system to the desired effect on the dynamical system comprises calculating respective differences between the respective corresponding effects on the dynamical system and the desired effect on the dynamical system, wherein each respective corresponding effect on the dynamical system and the desired effect on the dynamical system is a probability distribution, and wherein the selected next system action is a system action associated with the calculated respective difference having the lowest value.

24. The system of claim 15 , wherein the step of comparing each respective corresponding effect on the dynamical system to the desired effect on the dynamical system comprises calculating respective differences between the respective corresponding effects on the dynamical system and the desired effect on the dynamical system, wherein each respective corresponding effect on the dynamical system and the desired effect on the dynamical system is a probability distribution, and wherein the selected next system action is a system action associated with the calculated respective difference having the lowest value.