Method and apparatus for correlation of events in a distributed multi-system computing environment
A method and system is disclosed for monitoring an operation of a distributed data processing system. The system can include a plurality of applications running on a plurality of host processors and communicating with one another, such as through a message passing technique. The method includes steps executed in individual ones of the plurality of applications, of (a) examining individual ones of generated API calls to determine if a particular API call meets predetermined API call criteria; (b) if a particular API call meets the predetermined API call criteria, storing all or a portion of the content of the API call as a stored event; (c) processing a plurality of the stored events to identify logically correlated events, such as those associated with a business transaction; and (d) displaying all or a portion of the stored API call content data for the logically correlated events.
1. A computer-implemented method for monitoring an operation of a
transaction processing system, comprising steps of:
intercepting a first Application Program Interface (API) call;
examining said first API call, and if said first API call meets predetermined API call criteria, storing all or a portion of a content of said first API call as a first stored event;
intercepting a second API call;
examining said second API call, and if said second API call meets said predetermined API call criteria, storing all or a portion of a content of said second API call as a second stored event;
determining that said first API call is a part of a same particular business transaction as said second API call if:
(a) said first stored event indicates said first API call sent a message, and said second stored event indicates said second API call received said message, or
(b) said first and second stored events indicate said first and second API calls were conducted in a same transactional unit of work; and
if said first API call is a part of said same particular business transaction as said second API call, employing all or a portion of said first and second stored events in a subsequent process.
2. A method as in claim 1 , wherein the API call criteria comprises system entity identity.
3. A method as in claim 1 , wherein the API call criteria comprises API name.
4. A method as in claim 1 , wherein the API call criteria comprises timing data.
5. A method as in claim 1 , wherein the API call criteria comprises restrictions on parameter values to the API call.
6. A method as in claim 1 , wherein the step of intercepting said first API call includes a step of operating a sensor that is automatically enabled for responding to an occurrence of an error condition or a change in program states or environments.
7. A method as in claim 1 , wherein the step of intercepting said first API call includes a step of operating a sensor that is automatically enabled upon an occurrence of at least one pre-programmed triggering event, the sensor thereafter capturing all event data that satisfies a specific data collection filter.
8. A method as in claim 1 , wherein the step of employing includes a step of processing the first and second stored events using a script.
9. A method as in claim 8 , wherein the script is a pre-programmed script.
10. A method as in claim 8 , wherein the script is automatically generated by entry of data to a plurality of fields on a presentation filter dialogue box.
11. A method as in claim 8 , wherein the script is an operator-defined script.
12. A method as in claim 1 ,
wherein the step of intercepting said first API call comprises steps of:
installing a sensor between an output of an application and a function call library for emulating, relative to the application, an interface to the function call library; and
storing the predetermined API call criteria in a memory that is accessible by said sensor, and
wherein the step of examining said first API call comprises:
determining if the first API call fulfills the predetermined API call criteria; and
if a match occurs, capturing data representing all or a portion of the content of the first API call and transmitting the captured data to a database for storage as said first stored event.
13. A method as in claim 1 , wherein the step of intercepting said first API call comprises steps of:
installing a sensor between an output of an application and a function call library for emulating, relative to the application, an interface to the function call library; and
programming the predetermined API call criteria into a memory that is accessible by said sensor.
14. A method as in claim 1 , wherein the step of determining includes a step of correlating an entry event with an exit event.
15. A method as in claim 1 , wherein the step of determining includes a step of correlating a message queue put event with a message queue get event.
16. A method as in claim 1 , wherein the step of storing all or a portion of said content of said first API call includes a step of storing said first stored event in an event database with a unique ID, and wherein the step of determining includes a step of locating said first stored event in the event database using the unique ID.
17. A method as in claim 1 , wherein the step of storing all or a portion of said content of said first API call stores an entry and an exit for said first API call as one event in an event database and provides a unique ID for said first stored event, such that a matching algorithm need search only one field.
18. A method as in claim 17 , wherein the step of storing all or a portion of said content of said first API call further constructs a most-recently-stored (MRS) events list for improving the performance of the matching algorithm.
19. A method as in claim 1 , wherein the step of storing all or a portion of said content of said first API call employs a record address as a stored event ID, and wherein the step of determining provides cursors to access events according to various criteria, including an ability to enumerate through events one at a time, without requiring that all events be read into memory.
20. A method as in claim 1 , wherein the step of storing all or a portion of said content of said first API call provides event relationship lookup records to assist the performance of the step of determining by providing fast access to a list of events with a same attribute value.
21. An analyzer system for monitoring operation of a transaction processing system, comprising:
a first programmable sensor for intercepting and examining a first Application Program Interface (API) call to determine if said first API call meets programmed API call criteria, said first sensor being responsive to a condition that if said first API call meets the programmed API call criteria, capturing all or a portion of a content of said first API call;
a second programmable sensor for intercepting and examining a second API call to determine if said second API call meets the programmed API call criteria, said second sensor being responsive to a condition that if said second API call meets the programmed API call criteria, capturing all or a portion of a content of said second API call;
an analyzer console bidirectionally coupled to said first and second sensors, said analyzer console comprising:
(i) an event database having inputs coupled to said first and second sensors for storing said captured content of said first and second API calls as first and second stored events, respectively;
(ii) a data manager coupled to said event database for determining that said first API call is a part of a same particular business transaction as said second API call if:
(a) said first stored event indicates said first API call sent a message, and said second stored event indicates said second API call received said message, or
(b) said first and second stored events indicate said first and second API calls were conducted in a same transactional unit of work; and
(iii) a user interface for displaying all or a portion of said first and second stored events, if said first API call is a part of said same particular business transaction as said second API call; and a module that employs all of a portion of said first and second stored events in a subsequent process.
22. An analyzer system as in claim 21 , wherein said API call criteria comprises at least one of a system entity identity, an API name, timing data, and restrictions on parameter values to the API call.
23. An analyzer system as in claim 21 , wherein said first sensor is automatically enabled for responding to an occurrence of an error condition or a change in program states or environments.
24. An analyzer system as in claim 21 ,
wherein said first sensor is automatically enabled upon an occurrence of at least one programmed triggering event, and
wherein the first sensor thereafter captures all event data that satisfies a specific data collection filter.
25. An analyzer system as in claim 21 ,
wherein said data manager processes said first and second stored events using one of a pre-programmed script or an operator-defined script, and
wherein the script can be automatically generated by entry of data to a plurality of fields on a presentation filter dialogue box.
26. An analyzer system as in claim 21 ,
wherein said first sensor is installed between an output of an application and a function call library for emulating, relative to the application, an interface to the function call library, and
wherein predetermined API call criteria are programmed into a memory that is accessible by said first sensor.
27. An analyzer system as in claim 21 , wherein if said first sensor determines that said first API call fulfills the programmed API call criteria, said first sensor transmits said captured content of said first API call to said event database for storage as said first stored event.
28. An analyzer system as in claim 21 , wherein said data manager operates on said first and second stored events to at least one of correlate an entry event with an exit event, and correlate a message queue put event with a message queue get event.
29. An analyzer system as in claim 21 , wherein said first stored event in said event database is provided a unique ID for locating said first stored event in the event database.
30. An analyzer system as in claim 21 , wherein said first stored event in said event database comprises an entry and an exit for said first API call and is identified with a unique ID, such that an event matching algorithm need search only one data field.
31. An analyzer system as in claim 21 , wherein said first stored event in said event database comprises technology neutral event information, and wherein said second stored event in said event database comprises technology specific event information.
32. An analyzer system as in claim 31 , further comprising a technology-specific module that is invoked as needed to interpret said technology specific event information.
33. An analyzer system for monitoring operation of a data processing system, comprising:
a sensor for
(a) intercepting a first Application Program Interface (API) call,
(b) examining said first API call, and
(c) capturing first data from said first API call, if said first API call meets a criterion; and an analyzer console including
(a) an event database having an input coupled to said sensor for storing said data,
(b) a data manager coupled to said event database for determining that said first API call is logically related to a second API call if:
(i) said first data indicates said first API call sent a message, and second data from said second API call indicates said second API call received said message, or
(ii) said first data indicates said first API call was conducted in a transactional unit of work, and said second data indicates said second API call was also conducted in said transactional unit of work, and
(c) a module that employs said first data and said second data in a subsequent process, if said first API call is logically related to said second API call.
34. The analyzer system of claim 33 ,
wherein said data processing system is a distributed data processing system that includes a first processor and a second processor,
wherein said first API call is invoked by a first process running on said first processor, and
wherein said second API call is invoked by a second process running on said second processor.
35. The analyzer system of claim 33 , wherein said subsequent process displays said data.
36. The analyzer system of claim 33 , wherein said criterion is programmable, and
wherein said analyzer console provides configuration data to said sensor for programming said criterion.