METHOD, APPARATUS, AND SYSTEM FOR ADAPTING THE PRESENTATION OF USER INTERFACE ELEMENTS BASED ON A CONTEXTUAL USER MODEL
A method, apparatus, and system for adapting the presentation of user interface elements based on a contextual user model includes using passive interaction data, such as gaze-tracking inputs and/or certain proximity inputs, to determine an aspect of the user's current interaction context (e.g., the user's current focus of attention or current hand position). User interface elements may be changed or relocated based on the user's current interaction context.
1 . An adaptive presentation module to adapt the presentation of user interface elements at a computing device, comprising executable instructions embodied in one or more machine-readable media, configured to:
access a contextual user model, the contextual user model comprising data relating to:
gaze-tracking inputs received at the computing device,
on-screen locations corresponding to each of the gaze-tracking inputs, and
user interface elements displayed at the on-screen locations;
determine an aspect of user attention based on the contextual user model; and
change the presentation of a current user interface element based on the determined aspect of user attention.
2 . The adaptive presentation module of claim 1 , wherein the current user interface element comprises one or more of an interactive control, text, graphics, video, and web content.
3 . The adaptive presentation module of claim 1 , configured to determine a duration of user attention to an on-screen location corresponding to a gaze-tracking input and change the presentation of the current user interface element based on the duration of user attention.
4 . The adaptive presentation module of claim 1 , configured to determine an aspect of user attention to a user interface element located at an on-screen location corresponding to a gaze-tracking input and change the presentation of the current user interface element based on the determined aspect of user attention to the user interface element.
5 . The adaptive presentation module of claim 1 , configured to change the on-screen location of the current user interface element based on the determined aspect of user attention.
6 . The adaptive presentation module of claim 1 , configured to infer a user-specific current interaction context and move the current user interface element to an area of current visual focus based on the inferred current interaction context.
7 . The adaptive presentation module of claim 1 , configured to infer a user-specific current interaction context and rearrange a plurality of user interface elements based on the inferred current interaction context.
8 . The adaptive presentation module of claim 1 , configured to change a visual characteristic of the current user interface element based on the determined aspect of user attention.
9 . The adaptive presentation module of claim 1 , configured to change an audio characteristic of the current user interface element based on the determined aspect of user attention.
10 . The adaptive presentation module of claim 1 , configured to change the interaction mode of the current user interface element based on the determined aspect of user attention.
11 . The adaptive presentation module of claim 1 , configured to change the topology of the current user interface element based on the determined aspect of user attention.
12 . A method for adapting the presentation of user interface elements at a computing device, comprising:
maintaining a contextual user model, the contextual user model comprising:
gaze-tracking data comprising one or more real-time sensor inputs indicative of user gaze in relation to on-screen locations of a display of the computing device; and
semantic descriptions of user interface elements displayed at the on-screen locations corresponding to the gaze-tracking data, the semantic descriptions comprising information about the user interface elements;
determining an aspect of user attention based on the contextual user model; and
changing the presentation of a current user interface element based on the determined aspect of user attention.
13 . A computing system comprising:
a display;
a sensor subsystem to obtain gaze-tracking data, the gaze-tracking data being indicative of a user's gaze in relation to the display;
one or more processors; and
one or more machine-readable media having stored therein a plurality of instructions that when executed by the processor cause the computing system to perform the method of claim 12 .
14 . An adaptive presentation module to adapt the presentation of user interface elements at a computing device, comprising executable instructions embodied in one or more machine-readable media, the adaptive presentation module configured to:
access a contextual user model, the contextual user model comprising data relating to:
proximity inputs received at the computing device, the proximity inputs indicting hand locations relative to the computing device,
on-screen locations corresponding to the proximity inputs,
user interface elements displayed at on-screen locations in the vicinity of the on-screen locations corresponding to the proximity inputs, and
semantic descriptions of the user interface elements;
determine an aspect of user interaction based on the contextual user model; and
change the presentation of a current user interface element based on the determined aspect of user interaction.
15 . The adaptive presentation module of claim 14 , wherein the current user interface element comprises one or more of an interactive control, text, graphics, video, and web content.
16 . The adaptive presentation module of claim 14 , configured to change the on-screen location of the current user interface element based on the determined aspect of user interaction.
17 . The adaptive presentation module of claim 16 , configured to access a locomotive interaction model to determine an amount of time associated with a hand movement and change the on-screen location of the current user interface element based on the amount of time associated with the hand movement.
18 . The adaptive presentation module of claim 16 , configured to access a locomotive interaction model to determine a relative distance from one hand location to another hand location and change the on-screen location of the current user interface element based on the relative distance.
19 . The adaptive presentation module of claim 14 , configured to obtain, from the contextual user model, data relating to gaze-tracking inputs received at the computing device, on-screen locations corresponding to each of the gaze-tracking inputs, and user interface elements displayed at the on-screen locations; determine an aspect of user attention based on the contextual user model; and change the presentation of the current user interface element based on the determined aspect of user attention and the determined aspect of user interaction.
20 . The adaptive presentation module of claim 14 , configured to change the interaction mode of the current user interface element based on the determined aspect of user interaction.
21 . The adaptive presentation module of claim 14 , configured to change the topology of the current user interface element based on the determined aspect of user interaction.
22 . A method for adapting the presentation of user interface elements at a computing device, comprising:
maintaining a contextual user model, the contextual user model comprising data relating to:
proximity inputs received at the computing device, the proximity inputs indicting hand locations relative to the computing device,
on-screen locations corresponding to the proximity inputs,
user interface elements displayed at on-screen locations in the vicinity of the on-screen locations corresponding to the proximity inputs, and
semantic descriptions of the user interface elements;
determining an aspect of user interaction based on the contextual user model; and
changing the presentation of a current user interface element based on the determined aspect of user interaction.
23 . A computing system comprising:
a display;
a sensor subsystem to obtain proximity inputs;
one or more processors; and
one or more machine-readable media having stored therein a plurality of instructions that when executed by the processor cause the computing system to perform the method of claim 22 .
24 . A method for adapting the presentation of user interface elements at a computing device, comprising:
maintaining a contextual user model, the contextual user model comprising:
passive interaction data comprising one or more real-time sensor inputs indicative of a passive user interaction in relation to on-screen locations of a display of the computing device, the passive user interaction being an interaction that does not result in an application event;
active interaction data relating to a user interaction that results in an application event; and
semantic descriptions of user interface elements displayed at the on-screen locations corresponding to the active and passive interaction data, the semantic descriptions comprising information about the user interface elements;
determining an aspect of user attention based on the contextual user model; and
changing the presentation of a current user interface element based on the determined aspect of user attention.