Method, system and apparatus for operating a device using contextual scripting
A system and method for interacting with a digital device by acquiring an image of a display of the device, deriving the functional state and context of the device by analyzing the acquired image and commanding the device to perform an action. the procedure described above may be repeated by a script or test procedure.
1. A system for interacting with a device having a display comprising:
an image acquisition device to acquire an image of the display of said device; and
a processor including:
a parser module to identify objects contained in said image,
an analyzer module to derive functional information pertaining to said device based on said displayed objects,
a selector module to select a command to be executed by said device based on said functional information and according to a test procedure, and
an instructor module to instruct said device to execute said selected command.
2. The system of claim 1 , wherein said command is to cause said device to perform a functionality selected from the group consisting of short message service, multimedia message service, electronic mail, instant messaging, mobile advertising, multimedia related applications, downloadable applications, network related applications.
3. The system of claim 1 , wherein said command is to cause said device to download a downloadable application and execute said downloadable application.
4. The system of claim 1 , wherein said command is to cause said device to execute a multimedia application.
5. The system of claim 1 , wherein said command is to cause said device to execute a network related application.
6. The system of claim 1 wherein said command is selected according to a test procedure, and according to a functional state of said device, and according to a functional context of said device.
7. The system of claim 1 , wherein said parser module is to identify at least some of said objects by grouping a plurality of structured objects into a contextual object.
8. The system of claim 1 , wherein said parser module is to identify at least some of said objects by transforming said objects into predefined objects based on a template.
9. The system of claim 1 , wherein said processor further includes a translator module to translate said selected command into a sequence of instructions according to a template, wherein said template is associated with said device.
10. The system of claim 9 , further comprising a memory to store a plurality of templates, each said template being associated with at least one respective device, and wherein said processor is to select said template based on a type of said device.
11. The system of claim 9 , further comprising a device communication means to communicate said sequence of instructions to the device for execution.
12. A method for interacting with a digital device comprising:
acquiring an image of a display of said device;
identifying objects displayed on said display of said device;
deriving functional information pertaining to said device based on said displayed objects;
selecting a command to be executed by said device based on said functional information and a test procedure; and
instructing said device to execute said command.
13. The method of claim 12 , wherein said command is to cause said device to perform an application selected from the group consisting of short message service, multimedia message service, electronic mail, instant messaging, and mobile advertising.
14. The method of claim 12 , wherein said command is to cause said device to download a downloadable application and execute said downloadable application.
15. The method of claim 12 , wherein said command is to cause said device to execute a multimedia application.
16. The method of claim 12 , wherein said command is to cause said device to execute a network related application.
17. The method of claim 12 wherein said command is selected according to a test procedure, and according to a functional state of said device and according to a functional context of said device.
18. The method of claim 12 , further comprising translating said selected command into a sequence of instructions according to a template, wherein said template is associated with said digital device.
19. The method of claim 18 , further comprising choosing said template from a plurality of templates corresponding to a respective plurality of digital devices.
20. The method of claim 18 , wherein identifying said objects further comprises transforming said objects to predefined objects according to information in said template.
21. The method of claim 12 , wherein identifying at least some of said objects comprises grouping a plurality of structured objects into a contextual object.