Determining user interaction with user interface elements
Systems and methods described herein may provide a system that adjusts a user interface (UI) of a game in a non-obtrusive way to a user. The system may determine the least obtrusive area(s) on the screen for a UI element game overlay to be presented, at least in part by determining an importance level for area on the screen containing a UI element. The importance level for an area on the screen may be determined using a generated heat map that indicates the importance values of various areas of the screen. The heat map may be generated with inputs including one or more of user interactions with UI elements, game context, UI element rate of change, user matching, and user preferences. The system may temporarily hide UI elements from display that are determined to be below a threshold importance level.
1 . A method, comprising:
initializing an application session as a service such that a user interface (UI) of the application session is displayed on a display of a device;
detecting a plurality of UI elements included in the UI;
receiving indications of user interactions with the plurality of UI elements;
determining a UI element type of a first UI element of the plurality of UI elements;
generating a heat map based on the user interactions with the plurality of UI elements and the UI element type of the first UI element, the heat map indicating an importance score of each area of a plurality of areas of the display of the device, the importance score being with respect to a user that provides the user interactions;
determining an area of the display having an importance score below a predetermined threshold value based on the heat map;
modifying the UI such that a UI overlay is displayed in the determined area and overlayed over the application session;
identifying a second UI element of the plurality of UI elements that is disposed in an area of the display having an importance score below a second predetermined threshold value based on the generated heat map; and
hiding the second UI element that is identified from presentation on the display.
2 . The method of claim 1 , further comprising:
receiving an indication of information to provide on the display overlaid on the application;
determining an area of the display that is free of overlap with one of the plurality of UI elements and sized to accommodate the UI overlay comprising the information; and
displaying the UI overlay in the determined area and overlayed over the application.
3 . The method of claim 1 , further comprising determining a transparency value for the UI overlay based on the frequency of user interaction of the determined area.
4 . The method of claim 1 , wherein hiding the second UI element includes determining regions of the display for which a UI layer is hidden, the UI layer including the plurality of UI elements.
5 . The method of claim 1 , further comprising displaying the second UI element that is hidden in response to receiving an indication of a user interaction with the area of the display including the second UI element that is hidden.
6 . The method of claim 1 , wherein detecting the plurality of UI elements includes detecting coordinates of each of the plurality of UI elements on the display.
7 . The method of claim 1 , wherein a first UI element changes over time during the application session, and wherein generating the heat map is further based on a rate of change of the first UI element.
8 . The method of claim 1 , wherein generating the heat map is further based on rules set by the user regarding at least one area of the plurality of areas of the display of the device.
9 . The method of claim 1 , wherein the application session is a gaming session, and wherein generating the heat map is further based on a game context of a game application rendered by the gaming session.
10 . The method of claim 1 , wherein the application session is a gaming session, and wherein the gaming session is initialized as the service such that a game application is displayed on the display of the device along with the UI.
11 . An information handling system, comprising:
a memory;
a processor coupled to the memory, wherein the processor is configured to perform steps comprising:
initializing an application session as a service such that a user interface (UI) of the application session is displayed on a display of a device;
detecting a plurality of UI elements included in the UI;
receiving indications of user interactions with the plurality of UI elements;
determining a UI element type of a first UI element of the plurality of UI elements;
generating a heat map based on the user interactions with the plurality of UI elements and the UI element type of the first UI element, the heat map indicating an importance of each area of a plurality of areas of the display of the device, the importance being with respect to a user that provides the user interactions;
determining an area of the display having an importance score below a predetermined threshold value based on the heat map;
modifying the UI such that a UI overlay is displayed in the determined area and overlayed over the application session;
identifying a second UI element of the plurality of UI elements that is disposed in an area of the display having an importance score below a second predetermined threshold value based on the generated heat map; and
hiding the second UI element that is identified from presentation on the display.
12 . The information handling system of claim 11 , the steps further comprising:
determining an area of the display that is free of overlap with one of the plurality of UI elements and sized to accommodate the UI overlay; and
displaying the UI overlay in the determined area and overlayed over the application.
13 . The information handling system of claim 11 , the steps further comprising determining a transparency value for the UI overlay based on the frequency of user interaction of the determined area.
14 . The information handling system of claim 11 , wherein hiding the second UI element includes determining regions of the display for which a UI layer is hidden, the UI layer including the plurality of UI elements.
15 . The information handling system of claim 11 , the steps further comprising displaying the second UI element that is hidden in response to receiving an indication of a user interaction with the area of the display including the second UI element that is hidden.
16 . A computer program product, comprising:
a non-transitory computer readable medium comprising code for performing steps comprising:
initializing an application session as a service such that a user interface (UI) of the application session is displayed on a display of a device;
detecting a plurality of UI elements included in the UI;
receiving indications of user interactions with the plurality of UI elements;
determining a UI element type of a first UI element of the plurality of UI elements;
generating a heat map based on the user interactions with the plurality of UI elements and the UI element type of the first UI element, the heat map indicating an importance of each area of a plurality of areas of the display of the device, the importance being with respect to a user that provides the user interactions;
determining an area of the display having an importance score below a predetermined threshold value based on the heat map;
modifying the UI such that a UI overlay is displayed in the determined area and overlayed over the application session;
identifying a second UI element of the plurality of UI elements that is disposed in an area of the display having an importance score below a second predetermined threshold value based on the generated heat map; and
hiding the second UI element that is identified from presentation on the display.
17 . The computer program product of claim 16 , wherein the medium further comprises code for performing steps comprising:
receiving an indication of information to provide on the display overlaid on the application;
determining an area of the display that is free of overlap with one of the plurality of UI elements and sized to accommodate the UI overlay comprising the information; and
displaying the UI overlay in the determined area and overlayed over the application.