Correlative monitoring, analysis, and control of multi-service, multi-network systems
Methods and devices for correlative monitoring, analysis, and control of multi-service, multi-network systems are provided herein. Exemplary methods may include generating a network topology graph of a metanetwork using traffic records obtained from the metanetwork, generating a causal digraph of the metanetwork using the traffic records and the network topology graph, and transmitting one or more of the network topology graph and the causal digraph to a computing system.
1. A method of operating a correlative monitoring, analysis and control (MAC) system in connection with a production metanetwork having a plurality of interconnected networks including an intranet, an extranet, and a public internet, comprising:
obtaining traffic records from tap nodes disposed on respective communication channels within the metanetwork, the traffic records generated from message segments obtained from the communication channels, the traffic records including time information, source/destination information, and message content taken from the message segments, the traffic records being divided into distinct probe traffic records and target traffic records;
generating a metanetwork traffic model for the metanetwork, the metanetwork traffic model including:
a network topology graph of the metanetwork, the network topology graph created using, as input traffic records, the traffic records obtained from the metanetwork; and
a network segment transit-time digraph of the metanetwork, the network-segment transit-time digraph created using the input traffic records and the network topology graph;
processing the input traffic records, the network topology graph and the network segment transit-time graph to generate causal traffic-record selections, each including a respective probe traffic-record and one or more respective target traffic-records, the processing including, for each probe traffic-record, selecting causally related target-traffic records for inclusion in a causal-record selection for the probe traffic-record;
processing each causal traffic-record selection to generate a corresponding augmented traffic record by augmenting the message content of the probe traffic-record with adidentifying information obtained from the message content of the respective target traffic-records of the causal traffic-record selection, the adidentifying information providing supplemental distinguishing information for the augmented traffic record; and
displaying respective renderings of the augmented traffic-records to an analyst user,
wherein (a) the probe traffic-records are provided by a first tap node in front of or behind a merchant web server and include a given probe traffic-record with a session identifier of a user session on the merchant web server, (b) the respective target traffic-records of the causal record selection containing the given probe traffic-record are provided by a second tap node tapping traffic to and from another server where the session identifier is associated with a resource identifier of a resource used in the session, and (c) the augmented traffic-record of the causal record selection includes (i) the session identifier from the given probe-traffic record, and (ii) as the adidentifying information, the resource identifier for the user session,
wherein the other server includes either (1) a merchant database where the session identifier is associated with an account identifier as the resource identifier for the user session, the account identifier being obtained from the target traffic-records and included as the adidentifying information in the causal record selection, or (2) a payment processor where the session identifier is associated with a purchase-card identifier as the resource identifier for the user session, the purchase-card identifier being obtained from the target traffic-records and included as the adidentifying information in the causal record selection.
2. The method according to claim 1 , wherein the traffic records comprise message segments obtained from node points within the metanetwork, wherein the node points are communicatively coupled with services of the metanetwork.
3. The method according to claim 1 , wherein generating the traffic records includes:
obtaining message segments from the metanetwork;
time stamping the message segments;
pre-filtering the time stamped message segments to determine a source, destination, and segment size of the time stamped message segments; and
assembling the time stamped message segments into a traffic record.
4. The method according to claim 3 , further comprising one or more of decrypting and decoding the time stamped message segments and performing deep packet inspection of the time stamped message segments.
5. The method according to claim 4 , further comprising determining a user identification associated with message segments that have been subjected to deep packet inspection.
6. The method according to claim 1 , further comprising modeling congestion delay for the metanetwork using the causal digraph and predicting a service delay by comparing a benchmark congestion model to a subsequent congestion model.
7. The method according to claim 1 , further comprising storing the network topology graph and the causal digraph in a record, the record being stored in a storage media.
8. A correlative monitoring, analysis and control (MAC) system for use in connection with a production metanetwork having a plurality of interconnected networks including an intranet, an extranet, and a public internet, comprising:
tap nodes disposed on respective communication channels within the metanetwork to generate traffic records from message segments obtained from the communication channels, the traffic records including time information, source/destination information, and message content taken from the message segments, the traffic records being divided into distinct probe traffic records and target traffic records; and
at least one computing device comprising a memory that includes executable instructions and a processor executing the instructions to:
generate a metanetwork traffic model for the metanetwork, the metanetwork traffic model including (1) a network topology graph of the metanetwork, the network topology graph created using, as input traffic records, the traffic records obtained from the metanetwork;
a network segment transit-time digraph of the metanetwork, the network-segment transit-time digraph created using the input traffic records;
executing an event etiologizer to process the input traffic records, the network topology graph and the network segment transit-time graph to generate causal traffic-record selections, each including a respective probe traffic-record and a respective target traffic-record, the event etiologizer including a causal traffic-record filter operative for each probe traffic-record to select causally related target-traffic records for inclusion in a causal-record selection for the probe traffic-record;
processing each causal traffic-record selection to generate a corresponding augmented traffic record by augmenting the message content of the probe traffic-record with adidentifying information obtained from the message content of the respective target traffic-records of the causal traffic-record selection, the adidentifying information providing supplemental distinguishing information for the augmented traffic record; and
displaying respective renderings of the augmented traffic-records to an analyst user,
wherein (a) the probe traffic-records are provided by a first tap node in front of or behind a merchant web server and include a given probe traffic-record with a session identifier of a user session on the merchant web server, (b) the respective target traffic-records of the causal record selection containing the given probe traffic-record are provided by a second tap node tapping traffic to and from another server where the session identifier is associated with a resource identifier of a resource used in the session, and (c) the augmented traffic-record of the causal record selection includes (i) the session identifier from the given probe-traffic record, and (ii) as the adidentifying information, the resource identifier for the user session
wherein the other server includes either (1) a merchant database where the session identifier is associated with an account identifier as the resource identifier for the user session, the account identifier being obtained from the target traffic-records and included as the adidentifying information in the causal record selection, or (2) a payment processor where the session identifier is associated with a purchase-card identifier as the resource identifier for the user session, the purchase-card identifier being obtained from the target traffic-records and included as the adidentifying information in the causal record selection.
9. The system according to claim 8 , wherein the tap nodes are operative, when generating the traffic records, to:
obtain message segments from the metanetwork via node points;
time stamp the message segments;
pre-filter the time stamped message segments to determine a source, destination, and segment size of the time stamped message segments; and
assemble the time stamped message segments into a traffic record.
10. The system according to claim 9 , wherein the processor further executes the instructions to one or more of decrypt and decode the time stamped message segments.
11. The system according to claim 9 , wherein the processor further executes the instructions to perform deep packet inspection of the time stamped message segments.
12. The system according to claim 8 , wherein the processor further executes the instructions to:
model congestion delay for at least a portion of the metanetwork using the causal digraph; and
predict delay for a message segment being transmitted through at least a portion of the metanetwork.
13. The system according to claim 8 , wherein the processor further executes the instructions to store the network topology graph and the causal digraph in a record, the record being stored in a storage media.
14. The system according to claim 8 , wherein the traffic records comprise message segments obtained from node points within the metanetwork, wherein the node points are communicatively coupled with services of the metanetwork.
15. The system according to claim 14 , wherein the node points comprise bidirectional taps that detect input and output message segments of the services and networks, wherein the system further comprises a traffic monitor that receives traffic records from the bidirectional taps.
16. The system according to claim 14 , wherein the node points comprise bidirectional shunts that divert input and output message segments of the services and networks to a traffic monitor.
17. The system according to claim 14 , wherein a node point comprises any of a pair of unidirectional taps, a bidirectional multitap, a multiplexing bidirectional tap, or combinations thereof.
18. The system according to claim 14 , wherein the metanetwork comprises any of a shared-hub network, a shared-bus network, a token-ring network, a daisy-chain network, a switched network comprising a port mirroring switch, a switched network comprising bidirectional shunts.
19. The system according to claim 14 , wherein the at least one computing system and the node points are in apposed communicative coupling with the metanetwork.
20. The system according to claim 14 , wherein the at least one computing system and the node points are in interposed communicative coupling with the metanetwork.
21. The system according to claim 8 , wherein the causal traffic-record filter of the event etiologizer includes:
a euchronological target-epoch estimator using information from the metanetwork traffic model to estimate a temporal boundary between assay traffic-records related euchronologically versus dyschronologically to a given probe traffic-record in a given temporal direction and output the result as probe-assay euchronological epoch;
a euchronological traffic-record filter that filters the assay traffic-records in accordance with the probe-assay euchronological epoch and the given temporal direction, the filtering including (a) for a forwards temporal direction, rejecting those assay traffic-records whose start-time occurs before the epoch while passing those as followers whose start-time occurs at or after the epoch, and (b) for a backwards temporal direction, rejecting those assay traffic-records whose start-time occurs after the epoch and passing those as leaders whose start-time occurs at or before the epoch; and
a target-coidentifier traffic-record filter that filters euchronological-assay traffic-records in accordance with a probe coidentifier, an assay coidentifier, and the given temporal direction, passing as the target traffic-records those assay traffic-records whose assay coidentifier matches the probe coidentifier, and rejecting those assay traffic-records whose coidentifiers do not match, where the assay coidentifier is extracted from the assay traffic-records in accordance with a target coidentifier field-descriptor, and the probe coidentifier is extracted from a corresponding probe traffic-record in accordance with a probe coidentifier field-descriptor.