Granted Patent B2

US 11,782,557 · App. 17/176,033 · Granted Oct 10, 2023

Inventors: Björn Thomas Eriksson (Tynningö, SE); Sven Robert Pettersson (Huddinge, SE); Stefan Johannes Holmgren (Sollentuna, SE); Xiatao Wang (Lidingö, SE); Rozita Teymourzadeh (San Jose, CA); Per Erik Lindström (Årsta, SE); Emil Anders Braide (Årsta, SE); Jonas Daniel Justus Hjelm (Bandhagen, SE); Erik Anders Claes Rosengren (Stockholm, SE)

Assignee: Neonode, Inc.

G06F3/0421G06F3/013G06F3/017G06F3/041G06F3/0412G09G5/00G06F2203/04103G06F2203/04104

Abstract

Claims (23)

1. A method for interacting with controls in a graphical user interface (GUI) comprising a plurality of GUI controls, the method comprising:

identify a GUI control that a user is gazing at, from among a plurality of GUI controls presented on a display;

detect user interface gestures that are members of a group consisting of: slide gestures, sweep gestures and spread gestures, performed by the user;

select the identified GUI control when the detected gesture is performed in an airspace away from the display, by applying the detected gesture to the identified GUI control; and

selectively ignore the identified GUI control when the detected gesture is performed by the user touching the display, by applying the detected gesture to a GUI control presented at the touched locations that is different than the identified GUI control.

2. The method of claim 1 , further comprising altering the identified GUI control, to indicate to the user that gestures performed away from the display will be applied to the altered GUI control.

3. The method of claim 2 , wherein said altering the identified GUI control comprises at least one member of the group consisting of highlighting the identified GUI control and enlarging the identified GUI control.

4. The method of claim 1 further comprising:

format the detected gesture in a first human interface device (HID) format when the identified GUI control is selected; and

format the detected gesture in a second HID format when the identified GUI control is selectively ignored.

5. The method of claim 1 wherein a determination as to whether or not the gesture is performed by the user touching the display, is based on a detection plane in which the gesture is detected.

6. The method of claim 1 wherein a determination as to whether or not the gesture is performed by the user touching the display is based on a distance away from said display at which the gesture is detected.

7. The method of claim 1 wherein the identified GUI control is a slider or a scroll bar.

8. A user interface system comprising:

a display presenting a plurality of graphical user interface (GUI) controls;

an eye tracker operable to identify a GUI control that a user is gazing at, from among the plurality of GUI controls on said display;

a gesture sensor operable to detect gestures, that are members of a group consisting of: slide gestures, sweep gestures and spread gestures, performed on said display and gestures performed in an airspace away from said display, and to dynamically configure output data pertaining to each detected gesture, whereby (i) when the detected gesture is performed in an airspace away from said display, output the gesture in a first human interface device (HID) format, and (ii) when the detected gesture is performed by the user touching said display, output the gesture in a second HID format;

a processor connected to said display, said eye tracker and said gesture sensor; and

a non-volatile storage medium storing instructions, which, when executed by said processor, cause said processor (i) to selectively combine outputs from said eye tracker and said gesture sensor to apply those gestures received from said gesture sensor in the first HID format to the GUI control identified by said eye tracker, and (ii) to selectively ignore outputs from said eye tracker to apply those gestures received from said gesture sensor in the second HID format to a different GUI control presented at the touched display locations.

9. The user interface system of claim 8 wherein said gesture sensor and said eye tracker are housed in a common housing and activated by a shared controller.

10. The user interface system of claim 8 wherein said gesture sensor determines whether the gesture is performed by the user touching said display, or whether the gesture is performed away from said display, based on a detection plane in which the gesture is detected by said gesture sensor.

11. The user interface system of claim 8 wherein said gesture sensor determines whether the gesture is performed by the user touching said display, or whether the gesture is performed away from said display, based on a distance away from said display at which the gesture is detected.

12. The user interface system of claim 8 wherein the identified GUI control is a slider or a scroll bar.

Continuity (33)