3D gaze control of robot for navigation and object manipulation
View Patent ↗Disclosed herein are devices, methods, and systems for controlling a robot or assistive device that allows the robot or device to find and manipulate objects in a real world environment. The disclosed devices, methods, and systems may, in many cases, receive control input through monitoring, tracking, and analyzing the 3D gaze of a user/controller. Using the described 3D eye tracking, a user can directly and naturally look at an object of interest in the real world, while the system monitors, tracks, and records the user's 3D gaze position. This information, in many cases, is then translated into input commands for the robot or assistive.
1. A method of creating control output signals from visual information comprising:
monitoring one or more eyes of a user gazing at a target object;
determining and measuring movement of the eye and pupil;
analyzing the movement to identify intentional movement and unintentional movement;
translating the movement into data describing three dimensional movement of the user's gaze;
translating the data from the intentional movement into an output signal; and thereby
creating an output signal.
2. The method of claim 1 , wherein the analyzing involves the use of fuzzy logic.
3. The method of claim 1 , wherein the output signal is transferred to a robot or assistive device.
4. The method of claim 3 , wherein the robot or assistive device is controlled by one or more output signals.
5. The method of claim 1 , wherein the monitoring is performed by an optical device.
6. The method of claim 1 , the left and right eyes of the user are monitored to calculate a visual axis for each eye.
7. The method of claim 6 , wherein an intersection of the two visual axes is determined.
8. The method of claim 7 , wherein the intersection is used to calculate a gaze vector.
9. The method of claim 8 , wherein the gaze vector is used to calculate a gaze distance along the gaze vector.
10. The method of claim 6 , wherein the shortest distance between non-intersecting visual axes is determined.
11. The method of claim 10 , wherein the intersection is used to calculate a gaze vector.
12. The method of claim 11 , wherein the gaze vector is used to calculate a gaze distance along the gaze vector.