Apparatus and methods for online training of robots
Robotic devices may be trained by a user guiding the robot along a target trajectory using a correction signal. A robotic device may comprise an adaptive controller configured to generate control commands based on one or more of the trainer input, sensory input, and/or performance measure. Training may comprise a plurality of trials. During an initial portion of a trial, the trainer may observe robot's operation and refrain from providing the training input to the robot. Upon observing a discrepancy between the target behavior and the actual behavior during the initial trial portion, the trainer may provide a teaching input (e.g., a correction signal) configured to affect robot's trajectory during subsequent trials. Upon completing a sufficient number of trials, the robot may be capable of navigating the trajectory in absence of the training input.
1. A robot for adaptive training in real time, comprising:
a memory comprising a plurality of computer readable instructions stored thereon; and
at least one processor configured to execute the plurality of computer readable instructions to,
measure, in real time, autonomy of a controller during a first time interval, the autonomy of the controller at the first time interval corresponding to a first value,
transmit at least one training input to the controller during the first time interval,
measure, in real time, autonomy of the controller during a second time interval different from the first time interval, the beginning of the second time interval coincides with the end of the first time interval, the autonomy of the controller at the second time interval corresponding to a second value greater than the first value, the second value corresponding to iterative learning being performed by the controller based on the at least one training input, the iterative learning based on a prior performance obtained for a preceding iteration, and
navigate the robot along a desired trajectory to perform a task based on the iterative learning performed by the controller.
2. A method of training a robot in real-time, comprising:
measuring autonomy of a controller during a first time interval, the autonomy of the controller at the first time interval corresponding to a first value;
transmitting at least one training input to the controller during the first time interval;
measuring autonomy of the controller during a second time interval different from the first time interval, the beginning of the second time interval coincides with the end of the first time interval, the autonomy of the controller at the second time interval corresponding to a second value greater than the first value, the second value corresponding to iterative learning being performed by the controller based on the at least one training input, the iterative learning based on a prior performance obtained for a preceding iteration; and
navigating the robot along a desired trajectory to perform a task based on the iterative learning performed by the controller.
3. The method of claim 2 , wherein the first time interval corresponds to operational phase and the second time interval corresponds to learning phase, the operational and learning phases coincide with each other.
4. The method of claim 2 , further comprising:
designating a first weighting parameter to measure the autonomy of the controller at the first time interval, the first weighting parameter corresponds to an initial value; and
designating a second weighting parameter to measure the autonomy of the controller at the second time interval, the second weighting parameter composed of at least one of the training input or an output by the controller.
5. The method of claim 2 , further comprising:
designating a first weighting parameter to measure of the autonomy of the controller at the first time interval, the first weighting parameter composed of at least one of the at least one training input or an output by the controller; and
designating a second weighting parameter to measure the autonomy of the controller at the second time interval, the second weighting parameter composed of solely of the output by the controller.
6. The method of claim 2 , wherein the iterative learning is performed by the controller in real time while incorporating the at least one training input.
7. The method of claim 2 , further comprising:
adjust the iterative learning by the controller during the second time interval so as to influence an output by the controller.
8. The robot of claim 1 , wherein the first time interval corresponds to operational phase and the second time interval corresponds to learning phase, the operational and learning phases coincide with each other.
9. The robot of claim 1 , wherein the at least one processor is further configured to execute the computer readable instructions to,
designate a first weighting parameter to measure the autonomy of the controller at the first time interval, the first weighting parameter corresponds to an initial value; and
designate a second weighting parameter to measure the autonomy of the controller at the second time interval, the second weighting parameter composed of at least one of the training input or an output by the controller.
10. The robot of claim 1 , wherein the at least one processor is further configured to execute the computer readable instructions to,
designate a first weighting parameter to measure the autonomy of the controller at the first time interval, the first weighting parameter composed of at least one of the at least one training input or an output by the controller; and
designate a second weighting parameter to measure the autonomy of the controller at the second time interval, the second weighting parameter composed of solely of the output by the controller.
11. The robot of claim 1 , wherein the iterative learning ls performed by the controller in real time while incorporating the at least one training input.
12. The robot of claim 1 , wherein the at least one processor is further configured to execute the computer readable instructions to,
adjust the iterative learning by the controller during the second time interval so as to influence an output by the controller.
13. A non-transitory computer readable medium having computer readable instructions stored thereon, that when executed by at least one processor coupled to a robot configure the at least one processor to,
measure autonomy of a controller during a first time interval, the autonomy of the controller at the first time interval corresponding to a first value;
transmit at least one training input to the controller during the first time interval;
measure autonomy of the controller during a second time interval different from the first time interval, the beginning of the second time interval coincides with the end of the first time interval, the autonomy of the controller at the second time interval corresponding to a second value greater than the first value, the second value corresponding to iterative learning being performed by the controller based on the at least one training input, the iterative learning based on a prior performance obtained for a preceding iteration; and
navigate the robot along a desired trajectory to perform a task based on the iterative learning performed by the controller.
14. The non-transitory computer readable medium of claim 13 , wherein the first time interval corresponds to operational phase and the second time interval corresponds to learning phase, the operational and learning phases coincide with each other.
15. The non-transitory computer readable medium of claim 13 , wherein the at least one processor is further configured to execute the computer readable instructions to,
designate a first weighting parameter to measure the autonomy of the controller at the first time interval, the first weighting parameter corresponds to an initial value; and
designate a second weighting parameter to measure the autonomy of the controller at the second time interval, the second weighting parameter composed of at least one of the training input or an output by the controller.
16. The non-transitory computer readable medium of claim 13 , wherein the at least one processor is further configured to execute the computer readable instructions to,
designate a first weighting parameter to measure the autonomy of the controller at the first time interval, the first weighting parameter composed of at least one of the at least one training input or an output by the controller; and
designate a second weighting parameter to measure the autonomy of the controller at the second time interval, the second weighting parameter composed of solely of the output by the controller.
17. The non-transitory computer readable medium of claim 13 , wherein the iterative learning is performed by the controller in real time while incorporating the at least one training input.
18. The non-transitory computer readable medium of claim 13 , wherein the at least one processor is further configured to execute the computer readable instructions to,
adjust the iterative learning by the controller during the second time interval so as to influence an output by the controller.