Automating interactions with software user interface
A method and apparatus applies an action to a software application by determining a target object for the input action. The determination of the target object is performed by identifying the target object through socially identifying object information relative to a reference object. Then, the input action is applied to the target object.
1. A computer-implemented method, comprising:
recording, in an interaction file, a description of a user action, wherein the user action comprises execution of a software function represented by an object in a graphical user interface (GUI) displayed on a monitor, and wherein the object is identified in the description according to its socially identifying information, wherein the socially identifying information of the object comprises:
a location of a label of the object, wherein the label is separate from the object in the GUI and the location of the label describes a position of the label in the GUI relative to the object;
a description of a relationship of the object to one or more peer objects in the GUI displayed on the monitor;
a type of a parent object of the object; and
a rank of the object relative to the one or more peer objects, wherein the rank indicates a location of the object in the GUI relative to the one or more peer objects; and
executing a script file to perform the recorded user action on the object, wherein the script file is derived from the interaction file.
2. The method of claim 1 , wherein a first computer is used to record the user action on a second computer.
3. The method of claim 1 , wherein the user action is randomly selected from a list of possible actions.
4. The method of claim 1 , wherein the interaction file and the script file are a same file.
5. The method of claim 1 , wherein the recorded user action comprises signature information of the object.
6. The method of claim 1 , wherein the recording comprises capturing user interactions with another graphical user interface displayed on the monitor.
7. One or more tangible computer-readable storage media storing computer-readable instructions that, when executed by a computer, cause the computer to perform a method, the method comprising:
recording, in an interaction file, a description of a user action, wherein the user action comprises execution of a software function represented by an object in a graphical user interface (GUI) displayed on a monitor, and wherein the object is identified in the description according to its socially identifying information, wherein the socially identifying information of the object comprises:
a location of a label of the object, wherein the label is separate from the object in the GUI and the location of the label describes a position of the label in the GUI relative to the object;
a description of a relationship of the object to one or more peer objects in the GUI displayed on the monitor;
a type of a parent object of the object; and
a rank of the object relative to the one or more peer objects, wherein the rank indicates a location of the object in the GUI relative to the one or more peer objects; and
executing a script file to perform the recorded user action on the object, wherein the script file is derived from the interaction file.
8. The one or more tangible computer-readable storage media of claim 7 , wherein the user action is randomly selected from a list of possible actions.
9. The one or more tangible computer-readable storage media of claim 7 , wherein the recorded user action comprises signature information of the object.
10. The one or more tangible computer-readable storage media of claim 7 , wherein the recording comprises capturing user interactions with a graphical user interface displayed on the monitor.
11. An apparatus comprising:
a processor; and
a computer-readable medium having stored thereon a plurality of instructions which, when executed, cause the processor to:
record, in an interaction file, a description of a user action, wherein the user action comprises execution of a software function represented by a target object in a graphical user interface (GUI) displayed on a monitor, and wherein the target object is identified in the description according to its socially identifying information, wherein the socially identifying information of the object comprises:
label signature information, the label signature information describing a location of a label of the target object, wherein the label is separate from the target object and the location of the label describes a position of the label in the GUI relative to the target object;
a description of a relationship of the target object to one or more peer objects in the GUI displayed on the monitor;
a type of a parent object of the target object; and
a rank of the target object relative to the one or more peer objects, wherein the rank indicates a location of the target object in the GUI relative to the one or more peer objects; and
execute a script file to perform the recorded user action on the target object, wherein the script file is derived from the interaction file.
12. The apparatus of claim 11 , wherein the recording comprises capturing user interactions with the GUI.
13. The apparatus of claim 11 , wherein the user action is randomly selected from a list of possible actions.
14. The apparatus of claim 11 , wherein the interaction file and the script file are a same file.
15. The apparatus of claim 11 , wherein the recorded user action comprises signature information of the object.