System and method for dynamic accessibility app experiences
A method and system of providing users with accessibility tools based on the user's behavior during their access of an application. The system identifies behavior that is potentially associated with a particular disability. In response, the system causes one or more accessibility tools configured to improve the application experience for users with the identified disability to be enabled. In addition, past in-app behavior of a user may be used to determine whether an accessibility tool should again be offered to the same user during subsequent application use.
1 . A method for automatically enabling accessibility tools during use of an application in response to device data, the method comprising:
receiving, from a user interface of an automated teller machine (ATM) and a camera associated with the automated teller machine during a first application access session, at least first input data describing behavior of a first user while using the automated teller machine during the first application access session;
detecting using the application which is running on the automated teller machine a first usage pattern in the first input data;
determining using the application that the first usage pattern is associated with a first disability type;
responsive to determining the first pattern of behavior is associated with the first disability type, automatically selecting a first user experience setting for the application that is best configured to accommodate the first disability type;
notifying the first user in a first audio message via a speaker in the ATM that trouble accessing the user interface has been identified and that the first user experience setting for the application will be adjusted in response to the identified trouble accessing the user interface;
asking the first user for permission using voice input to make the adjustment to the first user experience setting; and
adjusting using the application without further action by the first user, during the first application access session the first user experience setting, the first user experience setting adjusting existing content in the user interface.
2 . The method of claim 1 , wherein the first input data is one of a sequence of touchscreen taps.
3 . The method of claim 1 , wherein the first accessibility tool causes audio playback of text being presented by the application to occur.
4 . The method of claim 1 , wherein the first accessibility tool causes closed captioning of audio being accessed through the application to be displayed.
5 . The method of claim 1 , wherein the first accessibility tool causes a magnification of elements of a user interface for the application.
6 . The method of claim 1 , wherein the first accessibility tool causes an increase in the volume of audio being played by the application.
7 . The method of claim 1 , further comprising detecting a face of a user that is at a first distance from the camera, the first distance being different than an average distance for a population of users of the application.
8 . The method of claim 1 , wherein:
receiving the first input data further comprises receiving the first input data via a touchscreen associated with the ATM; and
the first usage pattern is associated with users experiencing hand tremors.
9 . The method of claim 1 , further comprising determining the first input data indicates an inability to distinguish certain color(s) by the first user, and the first user experience setting includes one or more of an activation of a dictation tool, modification of contrast level, presentation of content in grayscale, change to colors of text, providing speech-based image descriptions, audio playback of text, and enabling closed captioning.
10 . A method for automatically enabling accessibility tools during use of an application in response to device data representing user behavior, the method comprising:
receiving from a user interface on a first smartwatch, during an application usage session, at least first input data to a first smartwatch from a first user while using the first smartwatch during the application usage session, the application running on the first smartwatch;
classifying, using the application during the application usage session, the first input data as corresponding to a first pattern of behavior, the first pattern of behavior referring to one or more of repetitive touchscreen taps, touchscreen taps in a non-clickable region, repeated undo usage, repeated spelling mistakes, repeated incorrect inputs, squinting, tremors, face moving farther than average or closer than average to a display of the first smartwatch, inappropriate selections of elements of the display of a specific color range, and repeated playback of audio or video;
determining using the application that the first pattern of behavior is associated with a first disability type;
responsive to determining the first pattern of behavior is associated with the first disability type, automatically selecting and adjusting using the application a first user experience setting for the application that is best configured to accommodate the first disability type, the first user experience setting adjusting existing content in the first smartwatch;
notifying the first user in a first audio message via a speaker in the first smartwatch that first disability type has been identified and that the first user experience setting for the application will be adjusted in response to the identified first disability type;
asking the first user for permission using voice input to make the adjustment to the first user experience setting; and
adjusting using the application, without further action by the first user, the first user experience setting during the application usage session.
11 . The method of claim 10 , wherein the first input data is one of a sequence of touchscreen taps.
12 . The method of claim 10 , wherein the adjusted first user experience setting causes audio playback of text being presented by the application.
13 . The method of claim 10 , wherein the adjusted first user experience setting causes closed captioning of audio being accessed through the application to be displayed.
14 . The method of claim 10 , wherein the adjusted first user experience setting causes a magnification of elements of a user interface for the application.
15 . The method of claim 10 , wherein the adjusted first user experience setting causes an increase in the volume of audio being played by the application.
16 . The method of claim 10 , further comprising:
obtaining the first input data via a camera associated with the smartwatch; and
detecting a face of a user that is at a first distance from the camera, the first distance being different than an average distance for a population of users of the application.
17 . The method of claim 10 , further comprising:
obtaining the first input data via a touchscreen associated with the smartwatch; and
wherein the first pattern of behavior is associated with users experiencing hand tremors.
18 . A system for automatically enabling accessibility tools during use of an application in response to device data, the system comprising a processor and machine-readable media including instructions which, when executed by the processor, cause the processor to:
receive, from a user interface of an automated teller machine (ATM) and a camera associated with the automated teller machine during a first application access session, at least first input data describing behavior of a first user while using the automated teller machine during the first application access session;
detect using the application which is running on the automated teller machine a first usage pattern in the first input data;
determine using the application that the first usage pattern is associated with a first disability type;
responsive to determining the first pattern of behavior is associated with the first disability type, automatically select a first user experience setting for the application that is best configured to accommodate the first disability type;
notify the first user in a first audio message via a speaker in the ATM that trouble accessing the user interface has been identified and that the first user experience setting for the application will be adjusted in response to the identified trouble accessing the user interface;
ask the first user for permission using voice input to make the adjustment to the first user experience setting; and
adjust using the application without further action by the first user, during the first application access session the first user experience setting, the first user experience setting adjusting existing content in the user interface.
19 . The system of claim 18 , wherein the first accessibility tool causes audio playback of text being presented by the application to occur.
20 . The system of claim 18 , wherein the first accessibility tool causes a magnification of elements of a user interface for the application.