Query-based identification of user interface elements
View Patent ↗Described is the use of query identifiers that provide computer users a method of “bookmarking” (uniquely identifying) any user interface (UI) element in any UI framework. This is useful for testing programs, assistive technologies, and teaching activities. The unique query element identifier is identified for a target UI element to locate within a hierarchy of UI elements. For example, the element's Name, ClassName, Instance, AccessibleName, Role, ControlId and AutomationId may be used to develop the unique query element identifier. From the query elements in the hierarchy of UI elements, a query identifier is generated for the target UI element. The query identifier includes the query element identifier and possibly ancestor query element identifiers, with each ancestor query element identifier corresponding to a higher-level ancestor element relative to the target element in the hierarchy. The query identifier may be used to locate the target element for testing versions of application programs.
1. At least one computer-readable storage medium having stored computer-executable instructions which when executed by a system implement a method for identifying a user interface element even when the element has been moved within the user interface from one node to another node of an interface hierarchy, the method comprising:
determining a unique query element identifier for a target user interface element within a hierarchy of user interface elements;
generating a partial query identifier for the target user interface element, the partial query identifier having the query element identifier therein and fewer than all ancestor query element identifiers, each ancestor query element identifier corresponding to a higher-level ancestor element relative to the target element in the hierarchy;
using the partial query identifier to locate the target user interface element for testing a first version of the target application program by at least traversing the hierarchy in a search operation; and
maintaining the partial query identifier to search for the target user interface element for testing a second version of the target application program to determine where the target user interface element has moved within the user interface, from one node to another node of the hierarchy associated with the interface, and as distinguished between the first and second versions of the target application program.
2. The computer-readable storage medium of claim 1 wherein determining the unique query element identifier comprises, selecting at least one identifier of a set, the set comprising Name, ClassName, Instance, AccessibleName, Role, ControlId and AutomationId.
3. The computer-readable storage medium of claim 2 wherein the unique query element is composed of at least two identifiers of the set, each identifier connected by a logical operator.
4. The computer-readable storage medium of claim 1 wherein the Partial query identifier includes at least one attribute.
5. The computer-readable storage medium of claim 4 wherein the at least one attribute corresponds to at least one attribute of a set of attributes, the set including, a type of search attribute, a navigational attribute and a state attribute.
6. The computer-readable storage medium of claim 5 wherein the type of search attribute corresponds to at least one type of search of a set, the set containing, Window, MSAA, and UIA.
7. The computer-readable storage medium of claim 5 wherein the navigational attribute corresponds to a relationship-related attribute of a set, the set containing parent, sibling, child, next to and the target element's position in a sequence.
8. The computer-readable storage medium of claim 5 wherein the state attribute corresponds to at least one state-related attribute of a set, the set containing, selectable, selected, expanded, collapsed, checked, and unchecked.
9. The computer-readable storage medium of claim 1 having further computer executable instructions, comprising, dynamically assigning a QueryElement delimiter on a per string basis, from a selected character in the string.
10. In a computing environment, a method for identifying a user interface element even when the element has been moved within the user interface from one node to another node of an interface hierarchy comprising:
determining a unique query element identifier for a target user interface element within a hierarchy of user interface elements;
generating a partial query identifier for the target user interface element to test within the hierarchy of user interface elements wherein generating the partial query identifier comprises including the query element identifier therein and fewer than all ancestor query element identifiers, each ancestor query element identifier corresponding to a higher-level ancestor element relative to the target element in the hierarchy;
using the partial query identifier to locate the target element for testing a first version of the target application program by at least traversing the hierarchy in a search operation; and
maintaining the partial query identifier to search for the target element for testing a second version of the target application program to determine where the target user interface element has moved within the user interface, from one node to another node of the hierarchy associated with the interface, and as distinguished between the first and second versions of the target application program.
11. The method of claim 10 wherein generating the unique query element identifier comprises, selecting at least one identifier of a set, the set comprising Name, ClassName, Instance, AccessibleName, Role, ControlId and AutomationId.
12. The method of claim 11 wherein the unique query element is composed of at least two identifiers of the set, each identifier connected by a logical operator.
13. The method of claim 10 wherein the partial query identifier includes at least one attribute.
14. The method of claim 10 further comprising, dynamically assigning a QueryElement delimiter on a per string basis, from a selected character in the string.