System and method for determining application dependency paths in a data center
View Patent ↗A method and a system are disclosed for determining application dependency paths in a data center. The method and the system captures application traffic volume data on the servers with switches and monitoring agents; generates an application traffic matrix of all the components of the applications based on the application traffic volume data; estimates the number of the applications in the data center from the traffic matrix with a Rank Estimation via Singular Value Decomposition or Power Factorization Residue Errors process; and decomposes the traffic matrix into a first matrix and a second matrix with a non-negative matrix factorization process using the estimated number of applications. The first matrix represents a set of the components belonging to each of the applications and the second matrix represents the amount of traffic generated by each application over time. Any noise in the first and second matrices is removed with a concurrent volumes ratios based correlation process.
1. A method for determining application dependency paths in a data center having a plurality of servers and a plurality of different applications, each of the applications having a plurality of components, one or more of the components being shared with one or more of the other applications, the method comprising the steps of:
capturing application traffic volume data on the servers;
generating an application traffic matrix of all the components of the applications based on the application traffic volume data;
estimating the number of the applications in the data center from the application traffic matrix;
decomposing the application traffic matrix into a first matrix and a second matrix using the estimated number of applications, the first matrix representing a set of the components belonging to each of the applications and the second matrix representing the amount of traffic generated by each application over time; and
removing noise from the first and second matrices, wherein the step of removing noise is performed with a concurrent volume ratios based correlation process.
2. The method of claim 1 , wherein the step of capturing application traffic volume data on the servers is performed in spatial and temporal domains.
3. The method of claim 1 , wherein the step of generating the application traffic matrix comprises generating application connectivity graphs for groups of the applications, the connectivity graphs represented by the application traffic matrix.
4. The method of claim 1 , wherein the step of estimating the number of the applications is performed with a Rank Estimation via Singular Value Decomposition or Power Factorization Residue Errors process.
5. The method of claim 4 , wherein the Rank Estimation via Power Factorization Residue Errors process uses a Power Factorization process.
6. The method of claim 1 wherein the step of decomposing the traffic matrix is performed with a non-negative matrix factorization process.
7. A system for determining application dependency paths in a data center having a plurality of servers and a plurality of different applications, each of the applications having a plurality of components, one or more of the components being shared with one or more of the other applications, the system comprising:
a data collection component; and
a controller in data communication with the data collection component;
the data collection component comprising:
a plurality of switches each having a mirror port for capturing application traffic volume data on the servers; and
a monitoring agent for monitoring each of the mirror ports;
the controller comprising:
a preprocessing unit including a connectivity graphing unit for generating an application traffic matrix of all the components of the applications based on the application traffic volume data captured by the monitoring server;
an application decomposition unit for estimating the number of the applications in the data center from the application traffic matrix generated by the connectivity graphing unit and for decomposing the application traffic matrix into a first matrix and a second matrix using the estimated number of applications, the first matrix representing a set of the components belonging to each of the applications and the second matrix representing the amount of traffic generated by each application over time; and
a postprocessing unit for removing noise from the first and second matrices, wherein the postprocessing unit performs a concurrent volume ratios based correlation process to remove noise from the first and second matrices.
8. The system of claim 7 , wherein the switches capture the application traffic volume data in spatial and temporal domains.
9. The system of claim 7 , wherein the application traffic matrix represents application connectivity graphs for groups of the applications.
10. The system of claim 7 , wherein the application decomposition unit performs a Rank Estimation via Singular Value Decomposition or Power Factorization Residue Errors process to estimate the number of the applications in the data center.
11. The system of claim 10 , wherein the Rank Estimation via Power Factorization Residue Errors process uses a Power Factorization process.
12. The system of claim 7 , wherein the application decomposition unit performs a non-negative matrix factorization process to decompose the traffic matrix into the first and second matrices.