Software robots for programmatically controlling computer programs to perform tasks

1. A system, comprising:

at least one computer hardware processor; and

at least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by the at least one computer hardware processor, cause the at least one computer hardware processor to control a first application program to perform a first action at least in part by:

automatically accessing, in an object hierarchy comprising a plurality of objects corresponding to active graphical user interface (GUI) elements of the first application program, a first object corresponding to a first active GUI element of the first application program, the accessing comprising searching for the first object in the object hierarchy, the searching comprising:

searching for a second object in the object hierarchy;

modifying the object hierarchy, when the first application program's graphical user interface has changed, by adding one or more new objects to the object hierarchy as descendants of the second object and/or removing one or more objects from the object hierarchy to obtain a modified hierarchy; and

searching for the first object in the modified object hierarchy; and

automatically using the first object to cause, via the first active GUI element, the first application program to at least partially perform the first action.

2. The system of claim 1 , wherein searching for the first object is performed using a search string formatted in accordance with a search grammar.

3. The system of claim 2 , wherein the search grammar comprises a token operator, an action operator, and a relationship operator.

4. The system of claim 3 , wherein the search string includes the action operator.

5. The system of claim 4 , wherein the action operator indicates how the object hierarchy is to be modified during performance of the search.

6. The system of claim 1 , wherein modifying the object hierarchy comprises:

using the second object to cause the first application program to perform an action causing one or more new GUI elements of the first application program to become active; and

modifying the object hierarchy by adding the one or more new objects corresponding to the one or more new GUI elements as the descendants of the second object in the object hierarchy.

7. The system of claim 1 , wherein searching for the second object is performed using a search string formatted in accordance with a search grammar, the search string including an action operator.

8. The system of claim 1 , wherein the first active GUI element of the first application program is a visible GUI element of the first application program.

9. The system of claim 1 , wherein the first active GUI element of the first application program is a hidden GUI element of the first application program.

10. A non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by at least one hardware processor, cause the at least one hardware processor to perform a method for controlling at least one application program to perform a task, the method comprising:

controlling a first application program to perform a first action associated with the task at least in part by:

automatically accessing, in an object hierarchy comprising a plurality of objects corresponding to active graphical user interface (GUI) elements of the first application program, a first object corresponding to a first active GUI element of the first application program, the accessing comprising searching for the first object in the object hierarchy, the searching comprising:

searching for a second object in the object hierarchy;

modifying the object hierarchy, when the first application program's graphical user interface has changed, by adding one or more new objects to the object hierarchy as descendants of the second object and/or removing one or more objects from the object hierarchy to obtain a modified hierarchy; and

searching for the first object in the modified object hierarchy; and

automatically using the first object to cause, via the first active GUI element, the first application program to at least partially perform the first action.

11. The non-transitory computer-readable storage medium of claim 10 , wherein searching for the first object is performed using a search string formatted in accordance with a search grammar.

12. The non-transitory computer-readable storage medium of claim 11 , wherein the search grammar comprises a token operator, an action operator, and a relationship operator.

13. The non-transitory computer-readable storage medium of claim 12 , wherein the search string includes the action operator.

14. The non-transitory computer-readable storage medium of claim 13 , wherein the action operator indicates how the object hierarchy is to be modified during performance of the search.

15. The non-transitory computer-readable storage medium of claim 10 , wherein modifying the object hierarchy comprises:

using the second object to cause the first application program to perform an action causing one or more new GUI elements of the first application program to become active; and

modifying the object hierarchy by adding the one or more new objects corresponding to the one or more new GUI elements as the descendants of the second object in the object hierarchy.

16. A method for controlling at least one application program to perform a task, the method comprising:

using at least one computer hardware processor to perform:

controlling a first application program to perform a first action associated with the task at least in part by:

automatically accessing, in an object hierarchy comprising a plurality of objects corresponding to active graphical user interface (GUI) elements of the first application program, a first object corresponding to a first active GUI element of the first application program, the accessing comprising searching for the first object in the object hierarchy, the searching comprising:

searching for a second object in the object hierarchy;

modifying the object hierarchy, when the first application program's graphical user interface has changed, by adding one or more new objects to the object hierarchy as descendants of the second object and/or removing one or more objects from the object hierarchy to obtain a modified hierarchy; and

searching for the first object in the modified object hierarchy; and

automatically using the first object to cause, via the first active GUI element, the first application program to at least partially perform the first action.

17. The method of claim 16 , wherein searching for the first object is performed using a search string formatted in accordance with a search grammar.

18. The method of claim 17 , wherein the search grammar comprises a token operator, an action operator, and a relationship operator, and the search string includes the action operator.

19. The method of claim 16 , wherein modifying the object hierarchy comprises:

using the second object to cause the first application program to perform an action causing one or more new GUI elements of the first application program to become active; and

modifying the object hierarchy by adding the one or more new objects corresponding to the one or more new GUI elements as the descendants of the second object in the object hierarchy.

20. The method of claim 16 , wherein searching for the second object is performed using a search string formatted in accordance with a search grammar, the search string including an action operator.