Monitoring user activity and performance of computerized devices
View Patent ↗The presently disclosed subject matter includes a device, system and method for monitoring activity at a computerized device, the device running one or more processes, at least one of the processes executing one or more system events being part of an activity. An activity of interest can be identified if system events related to the activity of interest are identified. An activity is monitored at the device using API queries to obtain descriptive information of the at least one respective system event executed as part of the activity. Using non-intrusive monitoring methods which are based on API queries enables to reduce the potential of interference of the monitoring with applications running on the monitored device.
1. A computerized method of monitoring activity at a monitored computerized device, the monitored device running one or more processes, at least one of the processes executing one or more system events being part of an activity, the method comprising, using a computer processor for:
obtaining an execution plan, the executing plan is generated based on agent configuration data; the agent configuration data is indicative of at least one process of interest and at least one activity signature; the activity signature representing a respective activity of interest and comprises a group of one or more opcodes enabling to uniquely identify the respective activity of interest; each opcode represents a respective system event of interest and comprises information indicative of one or more conditions, enabling to identify the respective system event of interest;
the execution plan comprising, for at least one respective system event of interest, one or more assistive technology API queries; each query is generated based on a condition of a respective opcode and enables to obtain descriptive information of the at least one respective system event of interest;
responsive to detection of a given system event, using the execution plan for calling the one or more assistive technology API queries of the given system event and obtaining descriptive information of the given system event;
determining that the given system event is a system event of interest, if the descriptive information of the given system event meets the respective conditions of the respective opcode; and
determining that a given activity of interest has been executed if system events of interest represented by a group of respective opcodes in a signature representing the activity of interest, are identified.
2. The method according to claim 1 further comprising: obtaining the agent configuration data and generating the execution plan based on the obtained agent configuration data.
3. The method according to claim 1 wherein the agent configuration data further comprises query logic comprising rules for guiding the generation of the execution plan; wherein the rules are configured to reduce interference with processes running on the monitored computerized device.
4. The method according to claim 3 wherein the execution plan is generated as a hierarchical data structure; wherein the one or more assistive technology API queries are ordered in the hierarchical data structure based on their respective priority.
5. The method according to claim 1 further comprising:
registering one or more listeners configured to listen to system events generated by one or more processes of interest capturing occurrence of system events.
6. The method according to claim 1 further comprising:
if it is determined that the given system event is a raw HID system event, linking the given system event with at least one related assistive technology object;
using the execution plan and invoking the one or more assistive technology API queries with respect to the at least one related assistive technology object.
7. The method according to claim 6 , wherein associating the given system event comprises:
responsive to detection of a mouse click event, obtaining screen coordinates indicative of the mouse click location on the screen; and
using assistive technology API queries to identify at least one related assistive technology object occupying the respective coordinates at the time of the mouse click event.
8. The method according to claim 6 further comprising:
maintaining a record of the last keyboard focus; responsive to detection of a keyboard stroke event, using assistive technology API queries to identify at least one related assistive technology object on which the last keyboard focus was located.
9. The method according to claim 1 wherein the determining that a given activity of interest has been executed further comprises:
determining that system events of interest represented by the group of respective opcodes in the signature comply with respective signature conditions defining the interrelations between the system events.
10. The method according to claim 1 further comprising:
collecting performance data of the activity of interest, the collected performance data enabling to determine respective performance metrics.
11. The method according to claim 1 further comprising:
generating the at least one activity signature, comprising:
obtaining data indicative of at least one process of interest;
registering listeners for detecting occurrence of system events executed by the at least one process of interest;
responsive to detection of a system event, using assistive technology API queries for enriching event descriptive data;
selecting a group of one or more system events and generating respective opcodes for each system event in the group; and
generating a respective activity signature comprising the group of one or more opcodes of interest, the activity signature uniquely identifying a given activity.
12. The method according to claim 11 further comprising, generating activity signatures based on rules directed for reducing interference with other processes running on the monitored computerized device.
13. A monitored computerized device, the computerized device comprising at least one computer processor operatively connected to computer memory configured for running one or more processes, at least one of the processes executing one or more system events being part of an activity; the at least one computer processor being configured to:
obtain an execution plan, the executing plan is generated based on agent configuration data; the agent configuration data is indicative of at least one process of interest and at least one activity signature; the activity signature representing a respective activity of interest and comprises a group of one or more opcodes enabling to uniquely identify the respective activity of interest; each opcode represents a respective system event of interest and comprises information indicative of one or more conditions, enabling to identify the respective system event of interest;
the execution plan comprising, for at least one respective system event of interest, one or more assistive technology API queries; each query is generated based on a condition of a respective opcode and enables to obtain descriptive information of the at least one respective system event of interest;
responsive to detection of a given system event, utilize the execution plan for calling the one or more assistive technology API queries of the given system event and obtain descriptive information of the given system event;
determine that the given system event is a system event of interest, if the descriptive information of the given system event meets the respective conditions of the respective opcode; and
determine that a given activity of interest has been executed if system events of interest represented by a group of respective opcodes in a signature representing the activity of interest are identified.
14. The device according to claim 13 wherein the at least one computer processor is further configured to obtain the agent configuration data and generate the execution plan based on the obtained agent configuration data.
15. The device according to claim 14 wherein the agent configuration data further comprises query logic comprising rules for guiding the generation of the execution plan; wherein the rules are configured to reduce interference with processes running on the monitored device.
16. The device according to claim 15 wherein the execution plan is generated as a hierarchical data structure; wherein the one or more assistive technology API queries are ordered in the hierarchical data structure based on their respective priority.
17. The device according to claim 13 wherein the at least one computer processor is further configured to: register one or more listeners configured to listen to system events generated by one or more processes of interest capturing occurrence of system events.
18. The device according to claim 13 wherein the at least one computer processor is further configured to: determine that the given system event is a raw HID system event and link the given system event with at least one related assistive technology object; and
call the one or more assistive technology API queries with respect to the at least one related assistive technology object.
19. The device according to claim 18 , wherein the at least one computer processor is further configured to:
responsive to detection of a mouse click event, obtaining screen coordinates indicative of the mouse click location on the screen; and
using assistive technology API queries to identify at least one related assistive technology object occupying the respective coordinates at the time of the mouse click event; and linking the given system event to the identified assistive technology object.
20. The device according to claim 18 wherein the at least one computer processor is further configured to:
maintain a record of the last keyboard focus; responsive to detection of a keyboard stroke event, utilize assistive technology API queries to identify at least one related assistive technology object on which the last keyboard focus was located.
21. The device according to claim 13 wherein the at least one computer processor is further configured for determining that a given activity of interest has been executed to:
determine that system events of interest represented by the group of respective opcodes in the signature comply with respective signature conditions defining the interrelations between the system events.
22. The device according to claim 13 further comprising:
collecting performance data of the activity of interest, the collected performance data enabling to determine respective performance metrics.
23. The device according to claim 13 further comprising:
generating the at least one activity signature, comprising:
obtaining data indicative of at least one process of interest;
registering listeners for detecting occurrence of system events executed by the at least one process of interest;
responsive to detection of a system event, using assistive technology API queries for enriching event descriptive data;
selecting a group of one or more system events and generating respective opcodes for each system event in the group; and
generating a respective activity signature comprising the group of one or more opcodes of interest, the activity signature uniquely identifying a given activity.
24. The device according to claim 23 generating activity signatures based on rules directed for reducing interference with other processes running on the same device.
25. A system for monitoring performance of one or more monitored computerized devices connected to the system over a communication network; the system comprises at least one processing unit comprising computer memory operatively connected to at least one processing unit; the at least one processing unit is configured to:
deploy an agent to each one of the one or more monitored computerized devices;
the agent comprising instructions for monitoring activities on the computerized devices to:
obtain an execution plan, the executing plan is generated based on agent configuration data; the agent configuration data is indicative of at least one process of interest and at least one activity signature; the activity signature representing a respective activity of interest and comprises a group of one or more opcodes enabling to uniquely identify the respective activity of interest; each opcode represents a respective system event of interest and comprises information indicative of one or more conditions, enabling to identify the respective system event of interest;
the execution plan comprising, for at least one respective system event of interest one or more assistive technology API queries; each query is generated based on a condition of a respective opcode and enables to obtain descriptive information of the at least one respective system event of interest;
responsive to detection of a given system event, utilize the execution plan for calling the one or more assistive technology API queries of the given system event and obtaining descriptive information of the given system event;
determine that the given system event is a system event of interest, if the descriptive information of the given system event meets the respective conditions of the respective opcode; and
determine that a given activity of interest has been executed if system events of interest represented by a group of respective opcodes in a signature representing the activity of interest are identified;
the processing unit is further configured to receive from the one or more monitored computerized devices performance data of the activity of interest and utilize the collected performance data for determining respective performance metrics.
26. A program storage device readable by a machine, tangibly embodying a program of instructions executable by the machine to perform method steps of monitoring activity at a monitored computerized device, the monitored device running one or more processes, at least one of the processes executing one or more system events being part of an activity, the method comprising:
obtaining an execution plan, the executing plan is generated based on agent configuration data; the agent configuration data is indicative of at least one process of interest and at least one activity signature; the activity signature representing a respective activity of interest and comprises a group of one or more opcodes enabling to uniquely identify the respective activity of interest; each opcode represents a respective system event of interest and comprises information indicative of one or more conditions, enabling to identify the respective system event of interest;
the execution plan comprising, for at least one respective system event of interest one or more assistive technology API queries; each query is generated based on a condition of a respective opcode and enables to obtain descriptive information of the at least one respective system event of interest;
responsive to detection of a given system event, using the execution plan for calling the one or more assistive technology API queries of the given system event and obtaining descriptive information of the given system event;
determining that the given system event is a system event of interest, if the descriptive information of the given system event meets the respective conditions of the respective opcode; and
determining that a given activity of interest has been executed if system events of interest represented by a group of respective opcodes in a signature representing the activity of interest are identified.