Techniques for identifying user interface elements and systems and devices using the same
The embodiments of the present disclosure relate generally to techniques for identifying elements in a user interface (UI), and more particularly, techniques for determining UI elements selected on a contact-sensitive user interface and using those techniques to provide one or more haptic responses.
1. A method of creating machine-readable instructions for searching for an element of a graphical user interface (GUI), the method comprising:
identifying elements of a GUI responsive to processing a machine-readable definition for the GUI;
determining searchable regions of the GUI using coordinate data;
associating respective identified elements with respective groups for elements located in same ones of the searchable regions;
arranging the identified elements into a tree structure;
combining respective arranged identified elements that are associated with a same one of the respective groups into respective single leafs of the tree structure thereby containing within a single leaf in the tree structure the respective identified elements associated with a same group; and
creating a list of search instructions responsive to the tree structure, the search instructions being machine-readable, the search instructions having an ordering according to a sequential execution of the search instructions.
2. The method of claim 1 , comprising creating one or more group search instructions responsive to the respective groups.
3. The method of claim 2 , comprising:
grouping all identified elements by common features;
reordering the tree structure to ensure all identified elements may be traversed in a single pass; and
eliminating redundant search instructions responsive to the reordering.
4. The method of claim 3 , comprising traversing all nodes of the tree structure and inserting one or more search instructions into a search instruction list responsive to visiting a node.
5. The method of claim 4 , wherein the list of search instructions is in an electronic filed formatting in an extensible markup language (XML).
6. An apparatus comprising a non-transitory computer readable medium storing instructions that, upon execution by a computer, enable the computer to:
identify elements of a GUI responsive to processing a machine-readable definition for the GUI;
determine searchable regions of the GUI using coordinate data;
associate respective identified elements with respective groups for elements located in the same ones of the searchable region;
arrange the identified elements into a tree structure;
combine respective arranged identified elements that are associated with a same one of the respective groups into respective single leafs in the tree structure thereby containing within a single leaf in the tree structure the identified elements associated with a same group; and
create a list of search instructions responsive to the tree structure, the search instructions being machine-readable, the search instructions having an ordering according to a sequential execution of the search instructions.
7. The apparatus of claim 6 , wherein the instructions enable the computer to create one or more group search instructions responsive to the respective groups.
8. The apparatus of claim 7 , wherein the instructions enable the computer to:
group all identified elements by common features;
reorder the tree structure to ensure all identified elements may be traversed in a single pass; and
eliminate redundant search instructions responsive to the reordering.
9. The apparatus of claim 8 , wherein the instructions enable the computer to traverse all nodes of the tree structure and insert one or more search instructions into a search instruction list responsive to a visit to a node.
10. The apparatus of claim 9 , wherein the list of search instructions is in an electronic file formatted according to an extensible markup language (XML).