Automated personalized feedback for interactive learning applications
A robot-training system permits a user touch, click on or otherwise select items from a display projected in the actual workspace in order to define task goals and constraints for the robot. A planning procedure responds to task definitions and constraints, and creates a sequence of robot instructions implementing the defined tasks.
1. A method of training a robot situated in a workspace, the method comprising the steps of:
computationally representing, by a controller, the robot in a 3D coordinate space corresponding to at least a portion of the workspace;
detecting, by the controller, a user gesture within the workspace, the gesture indicating a location, and based on the detected gesture, projecting an interface onto the workspace at the indicated location;
detecting, by the controller, gestural user selections of an element of the projected interface, the element of the projected interface corresponding to a robot control function involving manipulation of a physical object; and
causing the robot to execute the control function.
2. The method of claim 1 , wherein the user gesture is movement of a device, selections of elements being detected based on a position of the device relative to the projected interface.
3. The method of claim 2 , wherein the device is a wand having a fiducial thereon.
4. The method of claim 1 , wherein the control function includes at least one task goal and at least one task constraint.
5. The method of claim 4 , further comprising the step of analyzing a recorded image of the workspace to computationally define constraints associated with the task goals.
6. The method of claim 4 , further comprising the step of computing motion plans for the robot based on the task goals and the task constraints, the robot being caused to execute the control functions in accordance with the computed motion plans.
7. The method of claim 1 , further comprising the step of, following the gestural user selection of an interface element, detecting gestural user selection of an item within the workspace upon which the robot control function is to operate, the robot executing the control function on the selected item.
8. The method of claim 4 , wherein the at least one task goal comprises an object placement location.
9. The method of claim 8 , wherein the task constraint comprises an obstacle between an initial object location and the object placement location.
10. For use with a robot in a workspace including a projector and a camera, a robot controller comprising:
a processor;
a memory for storing images obtained by the camera and controller instructions executable by the processor to provide:
perception and interaction modules configured for (i) computationally representing the robot in a 3D coordinate spacer corresponding to at least a portion of the workspace, (ii) analyzing at least one first stored image to detect a user gesture within the workspace, the gesture indicating a location, (iii) based on the detected movement, causing the projector to project an interface onto the workspace at the indicated location, and (iv) analyzing at least one second stored image obtained by the camera subsequent to the at least one first stored image to detect a gestural user selection of an element of the projected interface, the element of the projected interface corresponding to a robot control function involving manipulation of a physical object; and
a planning module configured for computing robot instructions corresponding to the robot control function and causing the robot to execute the robot instructions.
11. The robot controller of claim 10 , wherein the robot controller is within the robot.
12. The robot controller of claim 10 , wherein the controller is external to the robot and communicates therewith via a network.
13. The robot controller of claim 10 , wherein the user gesture is movement of a device, selections of elements being detected based on a position of the device relative to the projected interface.
14. The robot controller of claim 10 , wherein the control function includes at least one task goal and at least one task constraint.
15. The robot controller of claim 14 , wherein the perception module is further configured for analyzing a stored image of the workspace to computationally define constraints associated with the task goals.
16. The robot controller of claim 14 , wherein the planning module is further for computing motion plans for the robot based on the task goals and the task constraints.
17. The robot controller of claim 10 , wherein the perception module is further configured for detecting gestural user selection of an item within the workspace upon which the robot control function is to operate.
18. The robot controller of claim 14 , wherein the at least one task goal comprises an object placement location.
19. The robot controller of claim 18 , wherein the task constraint comprises an obstacle between an initial object location and the object placement location.