Image based method and system for building object model and application states comparison and graphic-based interoperability with an application

1. A method for enabling graphic-based interoperability with a computer executed application, the method comprising, on a computer system operating as a client and displaying a graphical user interface (GUI), the GUI comprising control graphic items:

examining a graphical image of the GUI to determine if there has been a change over time in the GUI as displayed which updates a control graphic item;

if there has been a change over time in the GUI as displayed which updates a control graphic item, then taking an action, the action selected from:

updating properties of an object construct corresponding to the control graphic item; and

raising an event corresponding to an object construct corresponding to the control graphic item.

2. The method as in claim 1 , comprising examining the graphical image of the GUI upon an occurrence of the end of a time period, a detection of keyboard input, a detection of mouse input, or an indication that the GUI has changed.

3. The method as in claim 1 , wherein the control graphic item comprises one of a button, a link, a list box, a check box, a table, a drop-list, a combination box, a tab, and a slider.

4. The method as in claim 1 , comprising examining the graphical image of the GUI to identify within the GUI a control graphic item.

5. The method of claim 1 , comprising executing the computer executed application and a second program on the computer system, the second program to perform the examining.

6. The method of claim 5 , comprising executing server software modules on a computer remote from the computer system, server software modules to communicate with the executed application via the second program.

7. The method of claim 1 , wherein raising an event comprises determining that a state of the control graphic item has changed and notifying a process of the change.

8. A system for enabling graphic-based interoperability with a computer executed application, the system comprising:

a computer system operating as a client comprising a memory and a processor, the processor configured to:

display a graphical user interface (GUI), the GUI comprising control graphic items;

examine a graphical image of the GUI to determine if there has been a change over time in the GUI as displayed which updates a control graphic item;

if there has been a change over time in the GUI as displayed which updates a control graphic item, then take an action, the action selected from:

updating properties of an object construct corresponding to the control graphic item; and

raising an event corresponding to an object construct corresponding to the control graphic item.

9. The system as in claim 8 , the processor configured to examine the graphical image of the GUI upon an occurrence of the end of a time period, a detection of mouse input, a detection of keyboard input, or an indication that the GUI has changed.

10. The system as in claim 8 , wherein the control graphic item comprises one of a button, a link, a list box, a check box, a table, a drop-list, a combination box, a tab, and a slider.

11. The system as in claim 8 , the processor configured to examine the graphical image of the GUI to identify within the GUI a control graphic item.

12. The system as in claim 8 , the processor configured to execute the computer executed application and a second program on the computer system, the second program to perform the examining.

13. The system as in claim 12 , comprising a second computer remote from the computer system, the second computer configured to execute server software modules to communicate with the executed application via the second program.

14. The system of claim 8 , wherein raising an event comprises determining that a state of the control graphic item has changed and notifying a process of the change.

15. A method for communication between a first executing software program and a second executing software program, the method comprising:

the first executing software program obtaining a graphical image displayed by the second executing software program;

the first executing software program determining if there has been a change in a visual depiction of a graphical control within the obtained graphical image as compared to a previously cached version of the visual depiction of a graphical control in a graphical image displayed by the second executing software program; and

if there has been a change then notifying a process of the change.

16. The method as in claim 15 , comprising determining if there has been a change upon an occurrence of the end of a time period, a detection of keyboard input, a detection of mouse input, or an indication that the GUI has changed.

17. The method as in claim 15 , comprising simulating a functionality of the graphical control based on the type of the graphical control.

18. The method as in claim 15 , comprising examining the obtained graphical image to identify within the obtained graphical image a graphical control.

19. The method of claim 15 , comprising executing the first executing software program and the second executing software program on the same computer.

20. The method of claim 19 , comprising executing server software modules on a second computer remote from the computer, the server software modules to communicate with the second executing software program via the first executing program.

21. The method as in claim 1 , where the graphic control item is a user interface element.

22. The system of claim 8 , wherein the graphic control item is a user interface element.

23. The method as in claim 1 , wherein the GUI is executed by third-party software executing remotely from the process performing the examining and updating.

24. The system of claim 8 , wherein the GUI is executed by third-party software executing remotely from the process performing the examining and updating.