Automated characterization of network traffic
View Patent ↗Automated characterization of network traffic. A method may include compiling and/or receiving network traffic data based on the network traffic, preparing a network traffic characterization based on the network traffic data, and generating outgoing network traffic scripts based on the network traffic characterization. A method may also include capturing network traffic and generating test network traffic based on the scripts. A system on which the method may be executed may include one or more chassis and/or computing devices, each having one or more network cards. The chassis and/or computing devices may be connected to one or more networks and to one another. The networks may include a production network and a test network. A system may include a data collector, a characterization engine, and a script generator, and may also include a traffic generator.
1. A system to automatically generate outgoing network traffic based on observed production network traffic, the system comprising at least one network card, the network card including at least one processor and memory, the system further comprising:
a data collector to automatically capture production network traffic and to compile network traffic data based on the captured production network traffic, the network traffic data including: distribution data comprising at least one of protocol distribution data, length distribution data, and transaction distribution data; header information comprising at least one of a group of most common addresses, a group of most common ports, and most common header data; and payload data
a characterization engine to receive the production network traffic and the network traffic data from the data collector and to automatically prepare network traffic characterization data based on the production network traffic and the network traffic data, the network traffic characterization data including at least two of statistical indicators, a traffic model and traffic profile data,
a feedback unit included in the characterization engine, the feedback unit to communicate with the data collector to automatically adjust the network traffic data compiled based on earlier captured production network traffic including successively refining the network traffic data compiled
a script generator to receive the network traffic characterization data from the characterization engine and to automatically prepare a plurality of scripts based on the network traffic characterization data, each of the scripts associated with a type of network traffic modeled in the network traffic characterization data
a traffic generator to automatically generate test network traffic based on the scripts, including multiplexing in the test network traffic groupings of data units, each grouping representing the type of network traffic specified in the scripts
wherein the data collector, the characterization engine, the script generator and the traffic generator operate automatically without human intervention
a manager coupled with at least the data collector and the characterization engine, the manager to obtain statistical data from at least one of the data collector and the characterization engine and to provide an interface to allow a user to view information about the captured network traffic, including providing at least one of reports, log files, charts, and graphs showing at least one of the network traffic characterization data and the statistical data.
2. The system of claim 1 wherein the characterization engine comprises
a sort and statistics engine to prepare the statistical indicators based on the network traffic data
a modeling engine to prepare the traffic model based on the statistical indicators
a traffic profiler to generate the traffic profile data based on the traffic model and traffic mix profiles stored by the system.
3. The system of claim 2 wherein the characterization engine includes a model-tweaking unit to allow a user to edit the traffic model.
4. The system of claim 2 wherein the characterization engine includes a profile-editing unit to allow a user to edit the traffic profile data.
5. The system of claim 1 wherein the collector is configured to filter the network traffic based on filters and to translate the network traffic based on uniform format requirements.
6. The system of claim 5 wherein the filters are at least one of user defined and system defined.
7. The system of claim 1 wherein the characterization engine is configured to sanitize the network traffic data and the statistical indictors.
8. The system of claim 7 wherein to sanitize includes removing at least one of personal identifying information, passwords, bank account numbers, and credit card information.
9. The system of claim 1 wherein the data collector includes at least one data collection unit for each of a plurality of communications protocols supported by the system.
10. The system of claim 1 wherein the script generator includes at least one script generation unit for each of a plurality of communications protocols supported by the system.
11. The system of claim 1 wherein the traffic generator includes at least one traffic generation unit for each of a plurality of communications protocols supported by the system.
12. The system of claim 1 wherein the system supports a plurality of communications protocols.
13. The system of claim 12 wherein the plurality of communication protocols includes at least Ethernet, User Datagram Protocol (UDP), Transmission Control Protocol (TCP), and Hypertext Transfer Protocol (HTTP).
14. A system to automatically generate outgoing network traffic based on observed production network traffic, the system comprising at least one network card, the network card including at least one processor and memory, the system further comprising:
a data collector to automatically capture production network traffic and to compile network traffic data based on the captured network traffic, the network traffic data including distribution comprising: at least one of protocol distribution data, length distribution data, and transaction distribution data; header information comprising at least one of a group of most common addresses, a group of most common ports, and most common header data; and payload data
a feedback unit to receive captured production network traffic from the data collector and to communicate with the data collector to automatically adjust the network traffic data compiled based on the captured production network traffic including successively refining the network traffic data compiled
a sort and statistics engine to receive the network traffic data from the data collector and to automatically prepare statistical indicators based on the network traffic data
a modeling engine to receive the statistical indicators from the sort and statistics engine and to automatically prepare a traffic model based on the statistical indicators
a traffic profiler to receive the traffic model from the modeling engine and to automatically generate traffic profile data based on the traffic model and traffic mix profiles stored by the system
a script generator to receive the traffic profile data from the traffic profiler and to automatically prepare a plurality of scripts based on the traffic profile data, each of the scripts associated with a type of network traffic
a traffic generator to automatically generate test network traffic based on the scripts, including multiplexing in the test network traffic groupings of data units, each grouping representing the type of network traffic specified in the scripts
wherein the data collector, the feedback unit, the sort and statistics engine, the modeling engine, the traffic profiler, the script generator and the traffic generator operate automatically without human intervention
a manager coupled with the data collector, the feedback unit, the sort and statistics engine, the modeling engine, the traffic profiler, the script generator and the traffic generator, the manager to provide a first interface to allow a user to view information about the functioning of the system, including providing at least one of reports, log files, charts, and graphs showing at least one of the network traffic characterization data, the traffic profile data, the traffic model, and the statistical indicators.
15. The system of claim 14 wherein the data collector is configured to filter the network traffic based on filters and to translate the network traffic based on uniform format requirements.
16. The system of claim 14 wherein the modeling engine is configured to sanitize the network traffic data and the statistical indictors.
17. The system of claim 14 wherein to sanitize includes removing at least one of personal identifying information, passwords, bank account numbers, and credit card information.
18. The system of claim 14 wherein the modeling engine includes a model-tweaking unit to allow a user to edit the traffic model.
19. The system of claim 14 wherein the traffic profiler includes a profile-editing unit to allow a user to edit the traffic profile data.
20. The system of claim 14 wherein the script generator includes at least one script generation unit for each of the communications protocols supported by the system.
21. The system of claim 14 wherein the traffic generator includes at least one traffic generation unit for each of the communications protocols supported by the system.
22. The system of claim 14 wherein the system supports a plurality of communications protocols.
23. The system of claim 22 wherein the plurality of communication protocols includes at least Ethernet, User Datagram Protocol (UDP), Transmission Control Protocol (TCP), and Hypertext Transfer Protocol (HTTP).
24. A method to automatically generate outgoing network traffic based on observed production network traffic, the method performed by a network testing system, the method comprising:
the network testing system capturing production network traffic
the network testing system compiling network traffic data based on the captured production network traffic, the network traffic data including: distribution data comprising at least one of protocol distribution data, length distribution data, and transaction distribution data; header information comprising at least one of a group of most common addresses, a group of most common ports, and most common header data; and payload data
the network testing system automatically adjusting the network traffic data compiled based on earlier captured production network traffic including successively refining the network traffic data compiled
the network testing system preparing network traffic characterization data based on the production network traffic and the network traffic data the network traffic characterization data including at least two of statistical indicators, a traffic model and traffic profile data,
the network testing system generating a plurality of scripts based on the network traffic characterization data, each of the scripts associated with a type of network traffic modeled in the network traffic characterization data
the network testing system generating test network traffic based on the scripts, including multiplexing in the test network traffic groupings of data units, each grouping representing the type of network traffic specified in the scripts
wherein the capturing, the compiling, the preparing, the generating scripts, and the generating network traffic are performed by the network testing system automatically without human intervention.
25. The method of claim 24 wherein
the network testing system preparing network traffic characterization data includes
the network testing system preparing the statistical indicators based on the network traffic data
the network testing system preparing the traffic model based on the statistical indicators
the network testing system generating the traffic profile data based on the traffic model and system provided traffic mix profiles.
26. The method of claim 24 wherein
the network testing system capturing network traffic comprises the network testing system filtering the network traffic based on filters
the network testing system compiling network traffic data comprises the network testing system translating the network traffic based on uniform format requirements.
27. The method of claim 24 further comprising
the network testing system sanitizing the network traffic data and the statistical indicators.
28. The method of claim 27 wherein the network testing system sanitizing the network traffic data and the statistical indicators includes the network testing system removing at least one of personal identifying information, passwords, bank account numbers, and credit card information.
29. The method of claim 24 wherein the production network traffic includes a plurality of data units adhering to a plurality of communications protocols.
30. The method of claim 24 wherein the outgoing network traffic includes a plurality of data units adhering to a plurality of communications protocols.
31. The method of claim 30 wherein the plurality of communications protocols includes at least Ethernet, User Datagram Protocol (UDP), Transmission Control Protocol (TCP), and Hypertext Transfer Protocol (HTTP).
32. A non-transitory machine readable medium having instructions stored thereon which when executed cause a network testing system to perform operations comprising:
capturing production network traffic
compiling network traffic data based on the production network traffic, the network traffic data including: distribution data comprising at least one of protocol distribution data, length distribution data, and transaction distribution data; header information comprising at least one of a group of most common addresses, a group of most common ports, and most common header data; and payload data
automatically adjusting the network traffic data compiled based on earlier captured production network traffic, the network traffic characterization data including at least two of statistical indicators, a traffic model and traffic profile data including successively refining the network traffic data compiled
preparing network traffic characterization data based on the production network traffic and the network traffic data
generating a plurality of scripts based on the network traffic characterization data, each of the scripts associated with a type of network traffic modeled in the network traffic characterization data
generating test network traffic based on the scripts, including multiplexing in the test network traffic groupings of data units, each grouping representing the type of network traffic specified in the scripts
wherein the capturing, the compiling, the preparing, the generating scripts, and the generating network traffic are performed automatically without human intervention.
33. The non-transitory machine readable medium of claim 32 wherein
preparing network traffic characterization data includes
preparing the statistical indicators based on the network traffic data
preparing the traffic model based on the statistical indicators
generating the traffic profile data based on the traffic model and system provided traffic mix profiles.
34. The non-transitory machine readable medium of claim 32 wherein
the capturing network traffic comprises filtering the network traffic based on filters
the compiling network traffic data comprises translating the network traffic based on uniform format requirements.
35. The non-transitory machine readable medium of claim 32 having further instructions which when executed cause the processor to perform further operations comprising
sanitizing the network traffic data and the statistical indictors.
36. The non-transitory machine readable medium of claim 35 wherein the sanitizing includes removing at least one of personal identifying information, passwords, bank account numbers, and credit card information.
37. The non-transitory machine readable medium of claim 32 wherein the production network traffic includes a plurality of data units adhering to a plurality of communications protocols.
38. The non-transitory machine readable medium of claim 32 wherein the outgoing network traffic includes a plurality of data units adhering to a plurality of communications protocols.
39. The non-transitory machine readable medium of claim 38 wherein the plurality of communications protocols includes at least Ethernet, User Datagram Protocol (UDP), Transmission Control Protocol (TCP), and Hypertext Transfer Protocol (HTTP).
40. The non-transitory machine readable medium of claim 35 coupled with a network testing system.
41. The non-transitory machine readable medium of claim 40 wherein the network testing system is coupled to a production network.
42. The non-transitory machine readable medium of claim 41 wherein the network testing system is coupled to a test network.
43. A system to automatically generate outgoing network traffic scripts based on production network traffic, the system comprising at least one network card, the network card including at least one processor and memory, the system further comprising:
a collector to receive network traffic data based on the production network traffic
a characterization engine to receive the network traffic data from the collector and to prepare a network traffic characterization based on the network traffic data, the network traffic data including: distribution data comprising at least one of protocol distribution data, length distribution data, and transaction distribution data; header information comprising at least one of a group of most common addresses, a group of most common ports, and most common header data; and payload data
a feedback unit included in the characterization engine, the feedback unit to communicate with the collector to automatically adjust the network traffic data compiled based on earlier captured production network traffic including successively refining the network traffic data compiled
a script generator to receive the network traffic characterization from the characterization engine and to prepare a plurality of scripts based on the network traffic characterization, the network traffic characterization including at least two of statistical indicators, a traffic model and traffic profile data, each of the scripts associated with a type of network traffic modeled in the network traffic characterization
wherein the collector, the characterization engine and the script generator operate automatically with human intervention
a manager coupled with at least the collector and the characterization engine, the manager to obtain statistical data from at least one of the collector and the characterization engine and to provide an interface to allow a user to view information about the system, including providing at least one of reports, log files, charts, and graphs showing at least one of the network traffic characterization data, network traffic data and the statistical data
a traffic generator to automatically generate without human intervention test network traffic based on the scripts, including multiplexing in the test network traffic groupings of data units, each grouping representing the type of network traffic specified in the scripts.
44. The system of claim 43 wherein
the characterization engine comprises
a sort and statistics engine to prepare the statistical indicators based on the network traffic data
a modeling engine to prepare the traffic model based on the statistical indicators
a traffic profiler to generate the traffic profile data based on the traffic model and traffic mix profiles stored by the system.
45. The system of claim 44 wherein the modeling engine includes a model tweaking unit to allow a user to edit the traffic model.
46. The system of claim 44 wherein the traffic profiler includes a profile editing unit to allow a user to edit the traffic profile data.
47. The system of claim 43 wherein the data collector includes at least one data collection unit for each of a plurality of communications protocols supported by the system.
48. The system of claim 43 wherein the collector is configured to capture network traffic and to compile additional network traffic data based on the captured network traffic.
49. The system of claim 43 wherein the collector is configured to filter the network traffic data based on filters and to translate the network traffic data based on uniform format requirements.
50. The system of claim 43 wherein the script generator includes at least one script generation unit for each of a plurality of communications protocols supported by the system.
51. The system of claim 43 wherein the traffic generator includes at least one traffic generation unit for each of a plurality of communications protocols supported by the system.
52. The system of claim 43 wherein the system supports a plurality of communications protocols.
53. The system of claim 52 wherein the plurality of communication protocols includes at least Ethernet, User Datagram Protocol (UDP), Transmission Control Protocol (TCP), and Hypertext Transfer Protocol (HTTP).
54. A method to automatically generate outgoing network traffic scripts based on production network traffic, the method performed by a network testing system, the method comprising:
the network testing system receiving network traffic data based on production network traffic, the network traffic data including: distribution data comprising at least one of protocol distribution data, length distribution data, and transaction distribution data; header information comprising at least one of a group of most common addresses, a group of most common ports, and most common header data; and payload data
the network testing system automatically adjusting the network traffic data based on earlier captured production network traffic including successively refining the network traffic data compiled
the network testing system preparing a network traffic characterization based on the network traffic data, the network traffic characterization including at least two of statistical indicators, a traffic model and traffic profile data,
the network testing system automatically preparing without human intervention outgoing network traffic scripts based on the network traffic characterization, each of the scripts associated with a type of network traffic modeled in the network traffic characterization
the network testing system automatically generating without human intervention test network traffic based on the scripts, including multiplexing in the test network traffic groupings of data units, each grouping representing the type of network traffic specified in the scripts.
55. The method of claim 54 further comprising:
the network testing system automatically capturing without human intervention additional production network traffic
the network testing system automatically compiling without human intervention additional network traffic data based on the additional network traffic.
56. The method of claim 54 wherein
the network testing system preparing the network traffic characterization includes
the network testing system preparing the statistical indicators based on the network traffic data
the network testing system preparing the traffic model based on the statistical indicators
the network testing system generating the traffic profile data based on the traffic model and system provided traffic mix profiles.
57. The method of claim 54 wherein
the network testing system receiving network traffic data comprises
the network testing system filtering the network traffic data based on filters
the network testing system translating the network traffic based on uniform format requirements.
58. The method of claim 54 wherein the network traffic data is based on a plurality of data units adhering to a plurality of communications protocols.
59. The method of claim 55 wherein the test network traffic includes a plurality of data units adhering to a plurality of communications protocols.
60. The method of claim 54 wherein the plurality of communications protocols includes at least Ethernet, User Datagram Protocol (UDP), Transmission Control Protocol (TCP), and Hypertext Transfer Protocol (HTTP).