Method and system to identify GUI objects for non-markup-language-presented applications

1. A computer-executed method, comprising:

identifying, by a computer system, a set of target graphical user interface (GUI) objects in a non-markup-language-presented application, wherein a respective GUI object of the set of target GUI objects is not represented in a markup language;

generating a markup-language-based structure which includes one or more markup-language nodes that each specifies distinguishing attributes of a respective target GUI object in the non-markup-language-presented application, wherein generating the markup-language-based structure comprises:

determining a context root for the non-markup-language-presented application, wherein the context root serves as an entry point of a hierarchy of GUI objects and traversing the hierarchy of GUI objects starting from the context root while generating markup-language-based structure supports recording of user actions on the set of target GUI objects in the hierarchy of GUI objects;

converting the context root to a markup-language root node;

appending the context root to a new document; and

forming a hierarchy of nodes by performing operations including:

traversing the set of target GUI objects;

generating a markup-language node based on a respective target GUI object; and

adding the markup-language node to the new document;

generating addressing queries for selecting, from the markup-language-based structure, nodes corresponding to GUI objects that are to be identified;

generating an association map that maps each GUI object of the non-markup-language-presented application to a respective markup-language node;

generating a user action script, which involves:

responsive to a target GUI object being operated on, issuing a respective addressing query to the markup-language-based structure to select a markup-language node corresponding to the target GUI object being operated on, wherein each node of the markup-language-based structure is previously generated from a different corresponding GUI object of the non-markup-language-presented application;

identifying the target GUI object based on an entry of the association map corresponding to the selected markup-language node; and

recording, in the user action script, the addressing query and the user actions performed on the target GUI object; and

performing playback of the user action script to emulate recorded the user operations by executing one or more addressing queries against the markup-language-based structure, thereby identifying one or more GUI objects that have been previously operated and repeating the recorded user operations in the playback.

2. The method of claim 1 , wherein the non-markup-language-presented application is rendered on the GUI, without using a markup language representation for the set of target GUI objects.

3. The method of claim 1 , wherein converting the target GUI object to a markup-language node comprises:

naming the node with the name of the target GUI object; and setting attributes for the node based at least on class information and properties of the target GUI object.

4. The method of claim 1 , wherein at least one target GUI object is a container type object that contains other GUI objects.

5. The method of claim 1 , further comprising updating the markup-language-based structure and the association map to reflect changes to the non-markup-language-presented application's GUI objects.

6. The method of claim 1 , wherein generating a markup-language-based structure further comprises:

determining, without receiving user input at a respective GUI object, that the respective GUI object in the set of GUI objects allows a user to perform an action on the respective GUI object when interacting with a GUI; and

in response to the determining, generating a new node in the markup-language-based structure corresponding to the respective GUI object, wherein the respective GUI object does not correspond to any previously generated node in the markup-language-based structure.

7. A non-transitory computer-readable storage medium storing instructions which when executed by a particular machine cause the machine to perform a method, the method comprising:

identifying a set of target graphical user interface (GUI) objects in a non-markup-language-presented application, wherein a respective GUI object of the set of target GUI objects is not represented in a markup language;

generating a markup-language-based structure which includes one or more markup-language nodes that each specifies distinguishing attributes of a respective target GUI object in the non-markup-language-presented application, wherein generating the markup-language-based structure comprises:

determining a context root for the non-markup-language-presented application, wherein the context root serves as an entry point of a hierarchy of GUI objects and traversing the hierarchy of GUI objects starting from the context root while generating markup-language-based structure supports recording of user actions on the set of target GUI objects in the hierarchy of GUI objects;

converting the context root to a markup-language root node;

appending the context root to a new document; and

forming a hierarchy of nodes by performing operations including:

traversing the set of target GUI objects;

generating a markup-language node based on a respective target GUI object; and

adding the markup-language node to the new document;

generating addressing queries for selecting, from the markup-language-based structure, nodes corresponding to GUI objects that are to be identified;

generating an association map that maps each GUI object of the non-markup-language-presented application to a respective markup-language node;

generating a user action script, which involves:

responsive to a target GUI object being operated on, issuing a respective addressing query to the markup-language-based structure to select a markup-language node corresponding to the target GUI object being operated on, wherein each node of the markup-language-based structure is previously generated from a different corresponding GUI object of the non-markup-language-presented application;

identifying the target GUI object based on an entry of the association map corresponding to the selected markup-language node; and

recording, in the user action script, the addressing query and the user actions performed on the target GUI object; and

performing playback of the user action script to emulate the recorded user operations by executing one or more addressing queries against the markup-language-based structure, thereby identifying one or more GUI objects that have been previously operated and repeating the recorded user operations in the playback.

8. The computer-readable storage medium of claim 7 , wherein the non-markup-language-presented application is rendered on the GUI, without using a markup language representation for the set of target GUI objects.

9. The computer-readable storage medium of claim 7 , wherein converting the target GUI object to a markup-language node comprises;

naming the node with the name of the target GUI object; and setting attributes for the node based at least on class information and properties of the target GUI object.

10. The computer-readable storage medium of claim 7 , wherein at east one target GUI object is a container type object that contains other GUI objects.

11. A computer system, the system comprising:

a processor; a memory;

an identification mechanism configured to identify a set of target graphical user interface (GUI) objects in a non-markup-language-presented application, wherein a respective GUI object of the set of target GUI objects is not represented in a markup language;

a representation mechanism configured to generate a markup-language-based structure which includes one or more markup-language nodes that each specifies distinguishing attributes of a respective target GUI object in the non-markup-language-presented application, wherein generating the markup-language-based structure comprises:

determining a context root for the non-markup-language-presented application, wherein the context root serves as an entry point of a hierarchy of GUI objects and traversing the hierarchy of GUI objects starting from the context root while generating markup-language-based structure supports recording of user actions on the set of target GUI objects in the hierarchy of GUI objects;

converting the context root to a markup-language root node;

appending the context root to a new document; and

forming a hierarchy of nodes by performing operations including:

traversing the set of target GUI objects;

generating a markup-language node based on a respective target GUI object; and

adding the markup-language node to the new document;

a generation mechanism configured to generate addressing queries for selecting, from the markup-language-based structure, nodes corresponding to GUI objects that are to be identified;

wherein the representation mechanism is further configured to generate an association map that maps each GUI object of the non-markup-language-presented application to a respective markup-language node;

wherein the computer system further comprises an identification mechanism configured to generate a user action script, wherein generating the user action script involves:

responsive to a target GUI object being operated on, issuing a respective addressing query to the markup-language-based structure to select a markup-language node corresponding to the target GUI object being operated on, wherein each node of the markup-language-based structure is previously generated from a different corresponding GUI object of the non-markup-language-presented application

identifying the target GU object based on an entry of the association map corresponding to the selected markup-language node; and

recording, in the user action script, the addressing query and the user actions performed on the target GUI object; and

performing playback of the user action script to emulate the recorded user operations by executing one or more addressing queries against the markup-language-based structure, thereby identifying one or more GU objects that have been previously operated and repeating the recorded user operations in the playback.

12. The computer system of claim 11 , wherein the non-markup-language-presented application is rendered on the GUI, without using a markup language representation for the set of GUI objects.

13. The computer system of claim 11 , wherein the second conversion mechanism comprises:

a naming mechanism configured to name the node with the name of the target GUI object; and

an attribute-setting mechanism configured to set attributes for the node based at least on class information and properties of the target GUI object.

14. The computer system of clam 11 , wherein at least one target object is a container type object that contain other GUI objects.

15. The computer system of claim 11 , further comprising a receiving mechanism configured to receive a non-markup-language-presented application, wherein the non-markup-language-presented application includes a set of target GUI objects that are not represented in a markup language.

16. The computer system of claim 11 , further comprising a selecting mechanism configured to select a set of attributes that uniquely distinguish the target GUI object of the non-markup-language-presented application.