Database indexing in performance measurement systems
A performance measurement indexing system indexes a data store containing data entries indicative of message processing by an application. The application includes a plurality of checkpoints, and the data store contains data logged upon each message traversing the checkpoints in the application. The performance measurement indexing system determines which data entries relate to messages that satisfy a delay condition, and limits queries run on the data store to those data entries, thereby increasing the speed and efficiency with which queries can be serviced.
1. A computer implemented method comprising:
evaluating, by a processor, a data store coupled therewith, the data store storing data indicative of a traversal of each of a plurality of messages via each of a plurality of checkpoints of an application operative to process the plurality of messages upon a receipt of each thereof, to determine that a traversal latency of one or more of the plurality of messages between at least two checkpoints of the plurality of checkpoints satisfies a delay condition, and based thereon, modify the data indicative of the traversal of at least one of the at least two checkpoints by the one or more messages to indicate a delay; and
upon receiving, by the processor, a query of a first or second type, searching, by the processor based on the type of query, only data in the data store which indicates the delay when the query is of the first type or all of the data in the data store when the query is of the second type.
2. The computer implemented method of claim 1 , wherein the delay condition comprises one of: (i) the amount of time that elapses for a message to traverse the at least two checkpoints exceeds a threshold amount of time; (ii) the amount of time that elapses for a message to traverse the at least two checkpoints exceeds the amount of time that elapses for a message to traverse a different at least two checkpoints; (iii) the amount of time that elapses for a message to be processed by a portion of the application defined by the at least two checkpoints is greater than an amount of time between new messages arriving at that portion of the application; or (iv) if a number of messages awaiting processing by a portion of the application defined by the at least two checkpoints exceeds a threshold number of messages.
3. The computer implemented method of claim 1 , wherein the query of the first type is executed faster than the query of the second type.
4. The computer implemented method of claim 1 , wherein the evaluating further comprises determining an elapse of time between times when a received message traversed at least two of the at least two checkpoints.
5. The computer implemented method of claim 1 , wherein the traversal latency between the at least two checkpoints is determined based on the data stored in the data store indicative of the traversal of the one or more of the plurality of messages during the processing thereof of two or more of the plurality of checkpoints.
6. The computer implemented method of claim 1 , wherein the application is a first application, and wherein the data store further includes data indicative of message processing by a second application.
7. The computer implemented method of claim 6 , wherein the query of the first type comprises searching only the data indicative of the delay and indicative of messages processed by one of the first or second application.
8. The computer implemented method of claim 1 , wherein the query comprises one or more of: (i) determining percentiles associated with message processing; and (ii) determining the traversal latency between at least two of the plurality of checkpoints over a specified period of time.
9. The computer implemented method of claim 1 , wherein each message is an electronic data transaction request message, and wherein the application processes an electronic data transaction request message by determining whether an attempt to match an electronic data transaction request message with at least one previously received but unsatisfied electronic data transaction request message for a transaction which is counter thereto results in at least partial satisfaction of one or both of the electronic data transaction request message and the at least one previously received but unsatisfied electronic data transaction request message.
10. The computer implemented method of claim 1 , wherein the evaluating occurs only after all of the plurality of messages have been processed by the application.
11. The computer implemented method of claim 1 , wherein the query is received only after all of the plurality of messages have been processed by the application.
12. The computer implemented method of claim 1 , wherein the at least two checkpoints are not consecutive.
13. The computer implemented method of claim 1 , further comprising summarizing, by the processor, the result of the searching and presenting a visual representation thereof.
14. The computer implemented method of claim 13 , wherein the visual representation comprises a timeseries and percentile distribution.
15. A computer implemented method comprising:
evaluating, by a processor, data indicative of the processing of each of a plurality of messages by an application which causes each of the plurality of message to traverse at least one of a plurality of checkpoints of the application to determine that a latency between at least two checkpoints of the plurality of checkpoints satisfies a delay condition, and based thereon, modify the data based thereon; and
upon receiving, by the processor, a query of a first or second type, searching, by the processor based on the type of query, only the modified data or all of the data.
16. The computer implemented method of claim 15 , wherein the delay condition comprises one of: (i) the amount of time that elapses for a message to traverse the at least two checkpoints exceeds a threshold amount of time; (ii) the amount of time that elapses for a message to traverse the at least two checkpoints exceeds the amount of time that elapses for a message to traverse a different pair checkpoints; (iii) the amount of time that elapses for a message to be processed by a portion of the application defined by at least two of the plurality of checkpoints is greater than an amount of time between new messages arriving at that portion of the application; or (iv) if a number of messages awaiting processing by a portion of the application defined by at least two of the plurality of checkpoints exceeds a threshold number of messages.
17. The computer implemented method of claim 15 , wherein a search of only the modified data is executed faster than a search all of the data.
18. The computer implemented method of claim 15 , wherein the query comprises one or more of: (i) determining percentiles associated with message processing; and (ii) determining message processing latency between a pair of checkpoints from the plurality of checkpoints over a specified timeframe.
19. The computer implemented method of claim 15 , wherein the evaluating occurs only after all of the plurality of messages have been processed by the application.
20. The computer implemented method of claim 15 , wherein the query is received only after all of the plurality of messages have been processed by the application.
21. The computer implemented method of claim 15 , wherein the at least two checkpoints are not consecutive.
22. The computer implemented method of claim 15 , further comprising summarizing, by the processor, the result of the searching and presenting a visual representation thereof.
23. The computer implemented method of claim 22 , wherein the visual representation comprises a timeseries and percentile distribution.
24. A non-transitory computer-readable medium storing instructions which cause a processor to:
evaluate a data store coupled with the processor, the data store storing data indicative of a traversal of each of a plurality of messages via each of a plurality of checkpoints of an application operative to process the plurality of messages upon a receipt of each thereof, to determine that a traversal latency of one or more of the plurality of messages between at least two checkpoints of the plurality of checkpoints satisfies a delay condition, and based thereon, modify the data indicative of the traversal of at least one of the at least two checkpoints by the one or more messages to indicate a delay; and
upon receipt, by the processor, of a query of a first or second type, search, by the processor based on the type of query, only data in the data store which indicates the delay when the query is of the first type or all of the data in the data store when the query is of the second type.
25. The non-transitory computer-readable medium of claim 24 , wherein the delay condition comprises one of: (i) the amount of time that elapses for a message to traverse the at least two checkpoints exceeds a threshold amount of time; (ii) the amount of time that elapses for a message to traverse the at least two checkpoints exceeds the amount of time that elapses for a message to traverse a different pair checkpoints; (iii) the amount of time that elapses for a message to be processed by a portion of the application defined by the at least two checkpoints of the plurality checkpoints is greater than an amount of time between new messages arriving at that portion of the application; or (iv) if a number of messages awaiting processing by a portion of the application defined by the at least two checkpoints of the plurality of checkpoints exceeds a threshold number of messages.
26. The non-transitory computer-readable medium of claim 24 , wherein the query of the first type is faster than the query of the second type.
27. The non-transitory computer-readable medium of claim 24 , wherein the detection of whether the traversal latency between the at least checkpoints satisfies the delay condition comprises determining an elapse of time between times when the one or more messages traversed at least one of the at least two checkpoints.
28. The non-transitory computer-readable medium of claim 24 , wherein the instructions are further configured to cause the processor to determine the traversal latency between the at least two checkpoints based on data stored in the data store and indicative of times when the one or more messages traversed the at least two checkpoints.
29. The non-transitory computer-readable medium of claim 24 , wherein the query comprises one or more of: (i) a determination of percentiles associated with message processing; and (ii) a determination of the traversal latency between at least two of the plurality of checkpoints over a specified timeframe.
30. The non-transitory computer-readable medium of claim 24 , wherein the evaluation occurs only after all of the plurality of messages have been processed by the application.
31. The non-transitory computer-readable medium of claim 24 , wherein the query is received only after all of the plurality of messages have been processed by the application.
32. The non-transitory computer-readable medium of claim 24 , wherein the at least two checkpoints are not consecutive.
33. The non-transitory computer-readable medium of claim 24 , wherein the instructions are further configured to cause the processor to summarize the result of the search and present a visual representation thereof.
34. The non-transitory computer-readable medium of claim 33 , wherein the visual representation comprises a timeseries and percentile distribution.
35. A computer system comprising:
means for evaluating a data store, which stores data indicative of a traversal of each of a plurality of messages via each of a plurality of checkpoints of an application operative to process the plurality of messages upon a receipt of each thereof, to determine that a traversal latency of one or more of the plurality of messages between at least two checkpoints of the plurality of checkpoints satisfies a delay condition, and based thereon, modify the data indicative of the traversal of at least one of the at least two checkpoints by the one or more messages to indicate a delay; and
upon receiving a query of a first or second type, means for searching, based on the type of query, only data in the data store which indicates the delay when the query is of the first type or all of the data in the data store when the query is of the second type.