Database indexing in performance measurement systems
A performance measurement indexing system indexes a data store containing data entries indicative of message processing by an application. The application includes a plurality of checkpoints, and the data store contains data logged upon each message traversing the checkpoints in the application. The performance measurement indexing system determines which data entries relate to messages that satisfy a delay condition, and limits queries run on the data store to those data entries, thereby increasing the speed and efficiency with which queries can be serviced.
1 . A computer implemented method comprising:
evaluating, by a processor, a data store coupled therewith, the data store storing data entries tracing a progress of a traversal of each of a plurality of messages through a plurality of checkpoints of an application operative to process the plurality of messages upon a receipt of each thereof;
defining, by the processor, each of a plurality of portions of the application, by a pair of checkpoints of the plurality of checkpoints;
detecting, by the processor, that a message processing latency of one or more of the plurality of messages through a subset of the plurality of portions of the application satisfies a delay condition, and based thereon, augmenting each of the data entries associated with the pair of checkpoints defining the portions of the subset to indicate a delay; and
upon receiving, by the processor, a query that is limited to portions of the application that indicate the delay, searching, only the augmented data entries.
2 . The computer implemented method of claim 1 , wherein the delay condition is determined based on historical performance of messages that were previously processed through each of the portions of the subset.
3 . The computer implemented method of claim 1 , wherein the delay condition is configurable by a user.
4 . The computer implemented method of claim 1 , further comprising:
increasing, by the processor, a number of checkpoints within each of the portions of the subset.
5 . The computer implemented method of claim 1 , wherein the delay condition comprises one of: (i) an amount of time that elapses for a message to traverse a portion of the plurality of portions of the application exceeds a threshold amount of time; (ii) the amount of time that elapses for a message to traverse the portion of the application exceeds the amount of time that elapses for a message to traverse a different portion of the application; (iii) the amount of time that elapses for a message to be processed by the portion of the application is greater than an amount of time between new messages arriving at that portion of the application; or (iv) if a number of messages awaiting processing by the portion of the application exceeds a threshold number of messages.
6 . The computer implemented method of claim 1 , further comprising, upon receipt of a query to search the entire data store, searching, by the processor, all the data entries in the data store.
7 . The computer implemented method of claim 6 , wherein the limited query is executed faster than the query to search all the data entries.
8 . The computer implemented method of claim 1 , wherein the detecting further comprises determining an elapse of time when a received message traversed each of the portions of the subset.
9 . The computer implemented method of claim 1 , further comprising determining the message processing latency through each of the portions of the subset based on the data entries stored in the data store and indicative of times when the one or more messages traversed each of the portions of the subset.
10 . The computer implemented method of claim 1 , wherein the message processing latency of each of the portions of the subset is determined based on the data entries stored in the data store during the processing thereof of the pair of checkpoints.
11 . The computer implemented method of claim 1 , wherein the limited query comprises one or more of: (i) determining percentiles associated with message processing; and (ii) determining the message processing latency through each of the portions of the subset over a specified period of time.
12 . The computer implemented method of claim 1 , wherein each message is an electronic data transaction request message, and wherein the application processes an electronic data transaction request message by determining whether an attempt to match an electronic data transaction request message with at least one previously received but unsatisfied electronic data transaction request message for a transaction which is counter thereto results in at least partial satisfaction of one or both of the electronic data transaction request message and the at least one previously received but unsatisfied electronic data transaction request message.
13 . The computer implemented method of claim 1 , wherein the evaluating occurs only after all of the plurality of messages have been processed by the application.
14 . The computer implemented method of claim 1 , wherein the limited query is received only after all of the plurality of messages have been processed by the application.
15 . The computer implemented method of claim 1 , wherein the pair of checkpoints are not consecutive.
16 . The computer implemented method of claim 1 , further comprising modifying, by the processor, all of the data entries between the pair of checkpoints defining each of the portions of the subset to indicate a delay.
17 . A measurement indexing system comprising:
a processor;
a data store coupled with the processor, the data store configured to store data entries tracing a progress of a traversal of each of a plurality of messages through a plurality of checkpoints of an application operative to process the plurality of messages upon a receipt of each thereof;
a non-transitory computer-readable medium coupled with the processor, the non-transitory computer readable medium storing instructions which cause the processor to:
evaluate the data entries stored in the data store;
define, each of a plurality of portions of the application, by a pair of checkpoints of the plurality of checkpoints;
detect that a message processing latency of one or more of the plurality of messages through a subset of the plurality of portions of the application satisfies a delay condition, and based thereon, augment each of the data entries associated with the pair of checkpoints defining the portions of the subset to indicate a delay; and
upon receipt of a query that is limited to portions of the application that indicate the delay, search only the augmented data entries.
18 . The measurement indexing system of claim 17 , wherein the delay condition is determined based on historical performance of messages that were previously processed through each of the portions of the subset.
19 . The measurement indexing system of claim 17 , wherein the delay condition is configurable by a user.
20 . The measurement indexing system of claim 17 , wherein the instructions are further configured to cause the processor to increase a number of checkpoints within each of the portions of the subset.
21 . The measurement indexing system of claim 17 , wherein the delay condition comprises one of: (i) an amount of time that elapses for a message to traverse a portion of the plurality of portions of the application exceeds a threshold amount of time; (ii) the amount of time that elapses for a message to traverse the portion of the application exceeds the amount of time that elapses for a message to traverse a different portion of the application; (iii) the amount of time that elapses for a message to be processed by the portion of the application is greater than an amount of time between new messages arriving at that portion of the application; or (iv) if a number of messages awaiting processing by the portion of the application exceeds a threshold number of messages.
22 . The measurement indexing system of claim 17 , wherein the instructions further cause the processor to:
upon receipt of a query to search the entire data store, search all the data entries in the data store.
23 . The measurement indexing system of claim 22 , wherein the limited query is faster than the query to search all of the data entries.
24 . The measurement indexing system of claim 17 , wherein the detection of whether the message processing latency through each of the portions of the subset satisfies the delay condition comprises determination of an elapse of time between times when the one or more messages traversed each of the portions of the subset.
25 . The measurement indexing system of claim 17 , wherein the instructions are further configured to cause the processor to determine the message processing latency through each of the portions of the subset based on the data entries stored in the data store and indicative of times when the one or more messages traversed each of the portions of the subset.
26 . The measurement indexing system of claim 17 , wherein the message processing latency of each of the portions of the subset is determined based on the data entries stored in the data store during the processing thereof of the pair of checkpoints.
27 . The measurement indexing system of claim 17 , wherein the limited query comprises one or more of: (i) a determination of percentiles associated with message processing; and (ii) a determination of the message processing latency through each of the portions of the subset over a specified timeframe.
28 . The measurement indexing system of claim 17 , wherein each message is an electronic data transaction request message, and wherein the application processes an electronic data transaction request message by determining whether an attempt to match an electronic data transaction request message with at least one previously received but unsatisfied electronic data transaction request message for a transaction which is counter thereto results in at least partial satisfaction of one or both of the electronic data transaction request message and the at least one previously received but unsatisfied electronic data transaction request message.
29 . The measurement indexing system of claim 17 , wherein the evaluation occurs only after all of the plurality of messages have been processed by the application.
30 . The measurement indexing system of claim 17 , wherein the limited query is received only after all of the plurality of messages have been processed by the application.
31 . The measurement indexing system of claim 17 , wherein the pair of checkpoints are not consecutive.
32 . The measurement indexing system of claim 17 , wherein the instructions are further configured to cause the processor to modify all of the data entries between the pair of checkpoints defining each of the portions of the subset to indicate a delay.
33 . A measurement indexing system comprising:
means for evaluating a data store, which stores data entries tracing a progress of a traversal of each of a plurality of messages through a plurality of checkpoints of an application operative to process the plurality of messages upon a receipt of each thereof; and
means for defining each of a plurality of portions of the application by a pair of checkpoints of the plurality of checkpoints;
means for detecting that a message processing latency of one or more of the plurality of messages through a subset of the plurality of portions of the application satisfies a delay condition, and based thereon, augmenting each of the data entries associated with the pair of checkpoints defining the portions of the subset to indicate a delay; and
means for searching, upon receiving a query that is limited to portions of the application that indicate the delay, only the augmented data entries.