Graphical user interface input element identification
Systems, methods, and other embodiments associated with graphical user interface input element identification are described. One example system identifies a graphical user interface (GUI) element through which an input to an application was received based on comparing images generated from data provided by the application. Data that describes the GUI element may then be provided.
1. A tangible, non-transitory, computer-readable storage medium storing computer-executable instructions that when executed by a computer cause the computer to perform a method, the method comprising:
capturing a first image of a graphical user interface (GUI) of an application in response to detecting a user generated input to the application through the GUI;
identifying, based at least in part on the first image of the GUI, a GUI element through which the user generated input was received; and
providing data describing the GUI element.
2. The tangible, non-transitory, computer-readable medium of claim 1 , comprising capturing a second image of the GUI of the application in response to detecting the user generated input to the application through the GUI, where identifying the GUI element is also based on the second image of the GUI.
3. The tangible, non-transitory, computer-readable medium of claim 2 , where providing data identifying the GUI element comprises storing a test script, where the test script comprises data describing the user generated input and the data describing the GUI element.
4. The tangible, non-transitory, computer-readable medium of claim 1 , where the first image of the GUI shows a state of the GUI prior to the application receiving the user generated input.
5. The tangible, non-transitory, computer-readable medium of claim 2 , where the second image of the GUI shows a state of one or more of, the GUI during the application receiving the user generated input, and the GUI after the application has received the user generated input.
6. The tangible, non-transitory, computer-readable medium of claim 2 , where the first image and the second image are members of a series of images generated based on data provided by the application, where the application provides the data in response to inputs.
7. The tangible, non-transitory, computer-readable medium of claim 1 , where the user generated input is one or more of, a mouse down action, a mouse up action, an input from a peripheral device, and a keyboard button press.
8. The tangible, non-transitory, computer-readable medium of claim 3 , the method comprising:
controlling the application to receive a simulated input based on the test script.
9. The tangible, non-transitory, computer-readable medium of claim 8 , where controlling the application to receive a simulated input comprises:
capturing a third image of the GUI of the application;
identifying an input region of the GUI in the third image of the GUI based on the data describing the GUI element in the test script; and
providing the simulated input to the application through the input region, where the simulated input is computed as a function of the user generated input.
10. The tangible, non-transitory, computer-readable medium of claim 8 , where the method comprises controlling the application to receive a simulated input to analyze load on a device containing the application.
11. The tangible, non-transitory, computer-readable medium of claim 8 , where the method comprises controlling the application to receive a simulated input to evaluate responses of the application to various sequences of input.
12. The tangible, non-transitory, computer-readable medium of claim 1 , where the data describing the GUI element describes one or more of, the application, the GUI, hierarchy information associated with the GUI element, a type associated with the GUI element, text adjacent to the GUI element, a region of the GUI containing the GUI element, a specific location of the GUI element, ordering information about the GUI element, and a spatial relationship to a nearby GUI element.
13. A system, comprising:
a processor; and
a memory device comprising executable code that, when executed by the processor, causes the processor to:
store application testing instructions, where an application testing instruction identifies a graphical user interface (GUI) element and an input to provide to an application-under-test through the GUI element;
generate an application testing instruction based on a screen image of a GUI of the application-under-test and to store the application testing instruction in the data store, where the application testing instruction is generated upon detecting an input to the application-under-test through the GUI;
locate a GUI element that is to receive a test input, the GUI element being located in a screen image of a GUI of the application-under-test; and
provide input to the application-under-test through the GUI element that is to receive the test input, where the GUI element that is to receive the test input and the test input are identified in an application testing instruction from the data store.
14. The system of claim 13 , comprising:
a control logic to initiate a set of script replaying logics and to monitor information associated with one or more of, a test server running the application-under-test, the application-under-test, and members of the set of script replaying logics.
15. A system, comprising:
a processor; and
a memory device comprising executable code that, when executed by the processor, causes the processor to:
identify a graphical user interface (GUI) element through which an input to an application was received based on comparing images generated from data provided by the application; and
provide data describing the GUI element.