Arrangement, method and computer program for controlling a computer apparatus based on eye-tracking
View Patent ↗A computer based eye-tracking solution is disclosed. A computer apparatus is associated with one or more graphical displays (GUI components) that may be manipulated based on user-generated commands. An event engine is adapted to receive an eye-tracking data signal that describes a user's point of regard on the display. Based on the signal, the event engine is adapted to produce a set of non-cursor controlling event output signals, which influence the GUI-components. Each non-cursor controlling event output signal describes a particular aspect of the user's ocular activity in respect of the display. Initially, the proposed event engine receives a control signal request from each of the GUI-components. The control signal request defines a sub-set of the set of non-cursor controlling event output signals which is required by the particular GUI-component. The event engine delivers non-cursor controlling event output signals to the GUI-components in accordance with each respective control signal request.
1. An arrangement for controlling a computer apparatus associated with a graphical display, the display presenting a representation of a plurality of GUI-components that are configured to be manipulated based on user-generated commands, the arrangement comprising; a display; an event engine configured to receive an eye-tracking data signal describing a user's point of regard on the display, and at least based on the eye-tracking data signal, configured to produce a set of noncursor controlling event output signals influencing the GUI-components, each of the noncursor controlling event output signals describing a different aspect of the user's ocular activity in respect of the display, wherein the event engine is configured to:
receive a respective control signal request from each of the GUI-components, each respective control signal request defining a sub-set of the set of non-cursor controlling event output signals which is required by a corresponding one of the GUI-components to operate as intended,
produce the set of non-cursor controlling event output signals which is actually requested by the respective control signal request from each of the GUI-components, and
deliver the set of non-cursor controlling event output signals to the GUI-components in accordance with each respective control signal request.
2. The arrangement according to claim 1 , wherein the computer apparatus is configured to receive a cursor control signal, and control a graphical pointer on the display in response to the cursor control signal.
3. The arrangement according to claim 1 , wherein at least one of the GUI-components is configured to generate at least one respective output control signal upon a user manipulation of the component.
4. The arrangement according to claim 1 , wherein the event engine is configured to produce at least a first signal of the non-cursor controlling event output signals based on a dynamic development of the eye-tracking data signal.
5. The arrangement according to claim 4 , wherein the first signal represents a particular gaze pattern over the display.
6. The arrangement according to claim 5 , wherein at least one of the GUI-components is configured to interpret the first signal as an estimated intention of the user, and trigger a user manipulation of the component in response to the estimated intention.
7. The arrangement according to claim 5 , wherein at least one of the GUI-components is configured to interpret the first signal as an estimated attention level of the user, and trigger a user manipulation of the component in response to the estimated attention level.
8. The arrangement according to claim 5 , wherein at least one of the GUI-components is configured to interpret the first signal as a state-of-mind parameter of the user, and trigger a user manipulation of the component in response to the state-of-mind parameter.
9. The arrangement according to claim 1 , wherein the event engine is configured to receive at least one auxiliary input signal and produce the set of non-cursor controlling event output signals on the further basis of the at least one auxiliary input signal.
10. The arrangement according to claim 1 , wherein the at least one auxiliary input signal is originated from at least one of a button, a switch, a speech signal, a movement pattern of an input member, a gesture pattern, a facial expression and an EEG-signal.
11. A method of controlling a computer apparatus associated with a graphical display, the display representing a plurality of GUI-components that are configured to be manipulated based on user-generated commands, the method comprising:
receiving at an event engine an eye-tracking data signal which describes a user's point of regard on the display,
receiving at the event engine a respective control signal request from each of the GUI-components, the respective control signal request defining a sub-set of a set of noncursor controlling event output signals, the sub-set of the set of noncursor controlling event output signals being required by a corresponding one of the GUI-components to operate as intended,
producing the set of non-cursor controlling event output signals at least based on the eye-tracking data signal, the set of noncursor controlling event output signals influencing the GUI-components, each of the non-cursor controlling event output signals describing a different aspect of the user's ocular activity in respect of the display, the set of non-cursor controlling event output signals being actually requested by the respective control signal request from each of the GUI-components, and
delivering from the event engine the set of non-cursor controlling event output signals to the GUI-components in accordance with each respective control signal request.
12. The method according to claim 11 , further comprising receiving a cursor control signal, and controlling a graphical pointer on the display in response to the cursor control signal.
13. The method according to claim 11 , further comprising generating at least one output control signal by at least one of the GUI-components upon a user manipulation of the at least one of the GUI-components.
14. The method according to claim 11 , further comprising producing at least a first signal of the non-cursor controlling event output signals based on a dynamic development of the eye-tracking data signal.
15. The method according to claim 14 , wherein the first signal represents a particular gaze pattern over the display.
16. The method according to claim 15 , further comprising interpreting the first signal by at least one of the GUI-components as an estimated intention of the user, and triggering a user manipulation of the at least one of the GUI-components in response to the estimated intention.
17. The method according to claim 15 , further comprising interpreting the first signal by at least one of the GUI-components as an estimated attention level of the user, and triggering a user manipulation of the at least one of the GUI-components in response to the estimated attention level.
18. The method according to claim 15 , further comprising interpreting the first signal by at least one of the GUI-components as a state-of-mind parameter of the users, and triggering a user manipulation of the at least one of the GUI-components in response to the state-of-mind parameter.
19. The method according to claim 11 , further comprising: receiving at least one auxiliary input signal, and producing the set of non-cursor controlling event output signals on the further basis of the at least one auxiliary input signal.
20. The method according to claim 11 , wherein the at least one auxiliary input signal being originated from at least one of a button, a switch, a speech signal, a movement pattern of an input member, a gesture pattern, a facial expression and an EEG-signal.
21. A computer program embodied into the internal memory of a computer, comprising software for executing the method of claim 11 when said program is run on the computer.
22. A non-transitory computer readable medium, having a program recorded thereon for controlling a computer apparatus associated with a graphical display, the display representing a plurality of GUI-components that are to be manipulated based on user-generated commands, where the program operates as an event engine where the program includes computer readable program code portions stored therein, the computer readable program code portions comprising:
a first program code portion configured to receive an eye-tracking data signal which describes a user's point of regard on the display;
a second program code portion configured to receive a respective control signal request from each of the GUI-components, the respective control signal request defining a sub-set of a set of noncursor controlling event output signals, the sub-set of the set of noncursor controlling event output signals being required by a corresponding one of the GUI-components to operate as intended;
a third program code portion configured to produce the set of non-cursor controlling event output signals at least based on the eye-tracking data signal, the set of noncursor controlling event output signals influencing the GUI-components, each of the non-cursor controlling event output signals describing a different aspect of the user's ocular activity in respect of the display, the set of non-cursor controlling event output signals being actually requested by the respective control signal request from each of the GUI-components; and
a fourth program code portion configured to deliver the set of non-cursor controlling event output signals to the GUI-components in accordance with each respective control signal request.