Cloud service interdependency relationship detection
A computer system includes a processor, a memory, a data collector, a relationships analyzer, and a topological map generator. The data collector retrieves performance data in a specific set of performance categories for computing resources in a computing system for a time interval. The relationships analyzer, for each computing resource-to-computing resource pair in the computing system, performs a correlation analysis of the respective behavior values of the computing resources in the pair, and identifies the computing resource-to-computing resource pairs that have correlation values exceeding a pre-determined threshold level as having performance interdependencies. The topological map generator prepares an undirected graph of the computing resources that have performance interdependencies, and displays the undirected graph as a topographic map of the computing resources in the computing system.
1. A method for determining topological connections of a plurality of computing resources in a cloud computing system, the method comprising:
determining performances of each computing resource in the cloud computing system, each performance being determined independently of the determination of each performance of other computing resources;
for each computing resource-to-computing resource pair in the cloud computing system, performing a correlation analysis of the determined performances of the computing resources in the pair;
use the correlation analysis to determine topological connections of the plurality of computing resources in the cloud computing system by identifying each computing resource-to-computing resource pair that has a performance correlation value exceeding a pre-determined threshold level as being a topologically connected computing resource-to-computing resource pair;
assigning each performance correlation value exceeding the pre-determined threshold level to each corresponding topological connection between each corresponding computing resource-to-computing resource pair;
preparing an undirected graph of the topologically connected computing resource-to-computing resource pairs, based on the identifying of each computing resource-to-computing resource pair having the performance correlation value exceeding the pre-determined threshold level, and on the assigning of each performance correlation value exceeding the pre-determined threshold level to each corresponding topological connection; and
displaying, on a computer display, the undirected graph as a topographic map of the computing resources in the cloud computing system, including displaying the topological connections between computing resource-to-computing resource pairs, and providing the assigned performance correlation values for corresponding ones of the topological connections.
2. The method of claim 1 further comprising:
after performing the correlation analysis, performing a cross-correlation analysis of the determined performances of the computing resources in the pair to determine a time lag value that corresponds to a maximum performance correlation value.
3. The method of claim 2 further comprising:
based on the cross-correlation analysis, excluding computing resource-to-computing resource pairs that are cross-correlated with a non-zero lag and retaining only computing resource-to-computing resource pairs that are cross-correlated with a time lag of about zero for further processing.
4. The method of claim 2 , wherein preparing the undirected graph includes:
preparing an adjacency matrix representing a finite graph, based on the correlation analysis identifying the computing resource-to-computing resource pairs that have performance correlation values exceeding the pre-determined threshold level.
5. The method of claim 4 , wherein the pre-determined threshold level corresponds to a correlation value of about zero.
6. The method of claim 1 , wherein the performances include one or more of central processing unit (CPU) utilization, disk reads, disk writes, memory utilization, load average, processor queue length, context switches, and network bandwidth usage.
7. The method of claim 1 further comprising:
before performing the correlation analysis, performing dimensionality reduction to arrive at a single performance value for each of the computing resources in the cloud computing system.
8. The method of claim 1 further comprising:
updating the undirected graph to represent a current status of the computing resources in the cloud computing system.
9. A computer system comprising:
a processor;
a memory;
a data collector configured to:
determine a plurality of performances of each computing resource in a cloud computing system, each performance being determined independently of the determination of each performance of other computing resources;
a relationships analyzer configured to:
for each computing resource-to-computing resource pair in the cloud computing system, perform a correlation analysis of the determined performances of the computing resources in the pair; and
use the correlation analysis to determine topological connections of the plurality of computing resources in the cloud computing system by identifying each computing resource-to-computing resource pair that has a performance correlation value exceeding a pre-determined threshold level as being a topologically connected computing resource-to-computing resource pair;
assign each performance correlation value exceeding the pre-determined threshold level to each corresponding topological connection between each corresponding computing resource-to-computing resource pair;
and
a topological map generator configured to:
prepare an undirected graph of the topologically connected computing resource-to-computing resource pairs, based on the identifying of each computing resource-to-computing resource pair having the performance correlation value exceeding the pre-determined threshold level, and on the assigning of each performance correlation value exceeding the pre-determined threshold level to each corresponding topological connection; and
display, on a computer display, the undirected graph as a topographic map of the computing resources in the cloud computing system, including displaying the topological connections between computing resource-to-computing resource pairs, and providing the assigned performance correlation values for corresponding ones of the topological connections.
10. The computer system of claim 9 , wherein the relationships analyzer is configured to:
after performing the correlation analysis, perform a cross-correlation analysis of the determined performances of the computing resources in the pair to determine a time lag value that corresponds to a maximum performance correlation value.
11. The computer system of claim 10 , wherein the relationships analyzer is further configured to:
based on the cross-correlation analysis, exclude computing resource-to-computing resource pairs that are cross-correlated with a non-zero lag and retain only computing resource-to-computing resource pairs that are cross-correlated with a time lag of about zero for further processing.
12. The computer system of claim 10 , wherein the topological map generator is configured to:
prepare an adjacency matrix representing a finite graph, based on the correlation analysis identifying the computing resource-to-computing resource pairs that have correlation values exceeding the pre-determined threshold level.
13. The computer system of claim 12 , wherein the pre-determined threshold level corresponds to a correlation value of about zero.
14. The computer system of claim 9 , wherein the performances include one or more of CPU utilization, disk reads, disk writes, memory utilization, load average, processor queue length, context switches, and network bandwidth usage.
15. The computer system of claim 9 , wherein the relationships analyzer is configured to:
before performing the correlation analysis, perform dimensionality reduction to arrive at a single performance value for each of the computing resources in the cloud computing system.
16. The computer system of claim 9 , wherein the topological map generator is further configured to update the undirected graph to represent a current status of the computing resources in the cloud computing system.
17. A computer program product for determining topological connections of a plurality of computing resources in a cloud computing system, the computer program product including instructions recorded on a non-transitory computer-readable storage medium and configured to cause at least one processor to:
determine performance of each computing resource in the cloud computing system, each performance being determined independently of the determination of each performance of other computing resources;
for each computing resource-to-computing resource pair in the cloud computing system, perform a correlation analysis of the determined performances of the computing resources in the pair;
use the correlation analysis to determine topological connections of the plurality of computing resources in the cloud computing system by identifying each computing resource-to-computing resource pair that has a performance correlation value exceeding a pre-determined threshold level as being a topologically connected computing resource-to-computing resource pair;
assign each performance correlation value exceeding the pre-determined threshold level to each corresponding topological connection between each corresponding computing resource-to-computing resource pair;
prepare an undirected graph of the topologically connected computing resource-to-computing resource pairs, based on the identifying of each computing resource-to-computing resource pair having the performance correlation value exceeding the pre-determined threshold level, and on the assigning of each performance correlation value exceeding the pre-determined threshold level to each corresponding topological connection; and
display, on a computer display, the undirected graph as a topographic map of the computing resources in the computing system, including displaying the topological connections between computing resource-to-computing resource pairs, and providing the assigned performance correlation values for corresponding ones of the topological connections.
18. The computer program product of claim 17 , wherein the instructions, when executed, are further configured to cause the at least one processor to:
after performing a correlation analysis, perform a cross-correlation analysis of the determined performances of each computing resource-to-computing resource pair to determine a time lag value that corresponds to a maximum performance correlation value.
19. The computer program product of claim 18 , wherein the instructions, when executed, are further configured to cause the at least one processor to:
based on the cross-correlation analysis, exclude computing resource-to-computing resource pairs that are cross-correlated with a non-zero lag and retain only computing resource-to-computing resource pairs that are cross-correlated with a time lag of about zero for further processing.
20. The computer program product of claim 17 , wherein the instructions, when executed, are further configured to cause the at least one processor to:
when preparing the undirected graph, prepare an adjacency matrix representing a finite graph, based on the correlation analysis data identifying the computing resource-to-computing resource pairs that have performance correlation values exceeding the pre-determined threshold level.
21. The computer program product of claim 20 , wherein the pre-determined threshold level corresponds to a correlation value of about zero.
22. The computer program product of claim 17 , wherein the performances include one or more of CPU utilization, disk reads, disk writes, memory utilization, load average, processor queue length, context switches, and network bandwidth usage.
23. The computer program product of claim 17 , wherein the instructions, when executed, are further configured to cause the at least one processor to:
before performing the correlation analysis, perform dimensionality reduction to arrive at a single performance value for each of the computing resources in the cloud computing system.
24. The computer program product of claim 17 , wherein the instructions, when executed, are further configured to cause the at least one processor to:
update the undirected graph to represent a current status of the computing resources in the cloud computing system.
25. The method of claim 1 , wherein using the correlation analysis to indirectly determine the topological connections involves indirect detection of connections between computing resources by recognizing a topological connection between two computing resources upon detection of a performance or service interdependency relationship between the two computing resources.
26. The computer system of claim 9 , wherein the using the correlation analysis to indirectly determine the topological connections involves indirect detection of connections between computing resources by recognizing a topological connection between two computing resources upon detection of a performance or service interdependency relationship between the two computing resources.
27. The computer program product of claim 17 , wherein using the correlation analysis to indirectly determine the topological connections involves indirect detection of connections between computing resources by recognizing a topological connection between two computing resources upon detection of a performance or service interdependency relationship between the two computing resources.