Generation of test traffic configuration based on real-world traffic

1. A method for generating a test traffic configuration for testing a first network, the method comprising:

from a second network, receiving at least two data streams in two different formats, the data streams representing data traffic observed at one or more devices during a time interval in the second network that is a subset of data traffic processed by the one or more devices during the time interval;

normalizing the received data streams into a standardized format for consumption by a machine learning engine;

using the machine learning engine to analyze the set of normalized data streams in order to determine traffic patterns in the second network, wherein the analysis by the machine learning engine comprises:

based on traffic statistics for the time interval, determining characteristics of additional data traffic processed by the one or more devices during the time interval but not represented in the normalized data streams;

determining an optimal number of dimensions for the normalized data streams with data representing the additional data traffic;

reducing dimensionality of the normalized data streams with the data representing the additional data traffic to the determined optimal number of dimensions; and

using the dimension-reduced normalized data streams with the data representing the additional data traffic to determine the traffic patterns in the second network; and

generating the test traffic configuration for testing the first network by replicating the traffic patterns of the second network in the first network.

2. The method of claim 1 , wherein the second network is a production network with a deployed instance of an application and traffic-processing appliance that receives real-world data traffic.

3. The method of claim 2 , wherein the first network is a test network for testing a new release of the application and traffic-processing appliance.

4. The method of claim 2 , wherein the first network is a test network for testing another instance of the application and traffic-processing appliance.

5. The method of claim 1 , wherein determining the characteristics of the additional data traffic comprises using approximated optimal values to represent the additional data traffic.

6. The method of claim 1 , wherein reducing dimensionality of the normalized data streams with the data representing the additional data traffic comprises using a principal component analysis function.

7. The method of claim 1 , wherein using the dimension-reduced normalized data streams with the data representing the additional data traffic to determine the traffic patterns in the second network comprises using a clustering algorithm to determine the traffic patterns in the second network.

8. The method of claim 1 further comprising providing the generated test traffic configuration to a testing controller, wherein the testing controller uses the test traffic configuration to configure a set of test traffic sources.

9. The method of claim 8 , wherein the test traffic sources are network endpoints in the first network that generate data traffic to match the traffic patterns of the second network based on the test traffic configuration.

10. The method of claim 8 , wherein the testing controller and the set of test traffic sources are data compute nodes deployed in the first network.

11. The method of claim 1 , wherein at least a subset of the data streams are received from a set of network devices of the second network.

12. A non-transitory machine-readable medium storing a program which when executed by at least one processing unit generates a test traffic configuration for testing a first network, the program comprising sets of instructions for:

from a second network, receiving at least two data streams in two different formats, the data streams representing data traffic observed at one or more devices during a time interval in the second network that is a subset of data traffic processed by the one or more devices during the time interval;

normalizing the received data streams into a standardized format for consumption by a machine learning engine;

using the machine learning engine to analyze the set of normalized data streams in order to determine traffic patterns in the second network, wherein the analysis by the machine learning engine comprises:

based on traffic statistics for the time interval, determining characteristics of additional data traffic processed by the one or more devices during the time interval but not represented in the normalized data streams;

determining an optimal number of dimensions for the normalized data streams with data representing the additional data traffic;

reducing dimensionality of the normalized data streams with the data representing the additional data traffic to the determined optimal number of dimensions; and

using the dimension-reduced normalized data streams with the data representing the additional data traffic to determine the traffic patterns in the second network; and

generating the test traffic configuration for testing the first network by replicating the traffic patterns of the second network in the first network.

13. The non-transitory machine-readable medium of claim 12 , wherein:

the second network is a production network with a deployed instance of an application and traffic-processing appliance that receives real-world data traffic; and

the first network is a test network for testing the application and traffic-processing appliance.

14. The non-transitory machine-readable medium of claim 12 , wherein determining the characteristics of the additional data traffic comprises using approximated optimal values for the additional data traffic.

15. The non-transitory machine-readable medium of claim 12 , wherein the program further comprises a set of instructions for providing the generated test traffic configuration to a testing controller, wherein the testing controller uses the test traffic configuration to configure a set of network endpoints in the first network that generate data traffic to match the traffic patterns of the second network based on the test traffic configuration.

16. The method of claim 1 , wherein:

the normalized data streams comprise a set of parameters, each parameter corresponding to a definable parameter of the data traffic represented by the normalized data streams; and

after dimensionality of the normalized data streams with the data representing the additional data traffic is reduced, at least one resulting dimension corresponds to a combination of the parameters.

17. The method of claim 1 , wherein determining an optimal number of dimensions for the normalized data streams with data representing the additional data traffic comprises using maximum likelihood estimation.

18. The method of claim 1 , wherein reducing the dimensionality of the normalized data streams with data representing the additional data traffic avoids overfitting by the machine learning engine.

19. The non-transitory machine-readable medium of claim 12 , wherein using the dimension-reduced normalized data streams with the data representing the additional data traffic to determine the traffic patterns in the second network comprises using a clustering algorithm to determine the traffic patterns in the second network.

20. The non-transitory machine-readable medium of claim 12 , wherein:

the normalized data streams comprise a set of parameters, each parameter corresponding to a definable parameter of the data traffic represented by the normalized data streams; and

after dimensionality of the normalized data streams with the data representing the additional data traffic is reduced, at least one resulting dimension corresponds to a combination of the parameters.