Real world traffic
View Patent ↗Methods and apparatus for testing real-world performance of an system under test are disclosed. The system under test is loaded with interactive transactions. A realistic mix of network traffic is simulated on the communications network. The performance of the system under test under load is tested. Accordingly, a real-world test of the system under test is provided.
1. A method of testing real-world performance of a system under test coupled to a communications network, the method comprising
coupling a device to the communications network, the device comprising a chassis and one or more adapter cards, the adapter cards comprising hardware and software
the device setting up for simulation of a realistic mix of network traffic on the communications network
the device simulating the realistic mix of network traffic on the communications network
the device setting up for engaging in stateful TCP connections with the system under test
the device engaging in stateful TCP connections with the system under test concurrently with the step of simulating the realistic mix of network traffic on the communications network, resulting in additional network traffic on the communications network
the device measuring performance of the system under test for the stateful TCP connections under load of the simulated network traffic from the device.
2. The method of testing real-world performance of a system under test of claim 1 , wherein
the system under test comprises an application, the application operative on a server, the application for providing user-level interaction with plural client computers on the communications network.
3. The method of testing real-world performance of a system under test of claim 1 , wherein the system under test comprises a server load balancer.
4. The method of testing real-world performance of a system under test of claim 1 , wherein the system under test comprises a stateful network communications device.
5. The method of testing real-world performance of a system under test of claim 1 , wherein the simulated network traffic is generated by a stateless packet processor.
6. The method of testing real-world performance of a system under test of claim 1 , wherein the system under test comprises a stateful application which uses underlying services of TCP.
7. The method of testing real-world performance of a system under test of claim 6 , wherein the system under test comprises an HTTP server.
8. The method of testing real-world performance of a system under test of claim 6 , wherein the system under test comprises an FTP server.
9. The method of testing real-world performance of a system under test of claim 1 further comprising
modifying a behavior of the network traffic simulated by the device
continuing to engage in stateful TCP connections with the system under test
continuing to measure performance of the system under test for the stateful TCP connections.
10. The method of testing real-world performance of a system under test of claim 9 , the modifying step comprising using performance metrics based on the stateful TCP connections to modify the behavior of the simulated network traffic to more closely simulate a realistic mix of network traffic.
11. The method of testing real-world performance of a system under test of claim 10 wherein the performance metrics are selected from the group consisting of retransmission rate, fragmentation, packet sizes, and drop/reset rates.
12. The method of testing real-world performance of a system under test of claim 9 , the modifying step comprising a user using a control program to change the behavior of the simulated network traffic via a system interface.
13. The method of testing real-world performance of a system under test of claim 12 , the modifying step comprising the user managing multiple ports in a coordinated fashion.
14. An apparatus for testing real-world performance of a system under test coupled to a communications network, the apparatus comprising
a chassis
one or more adapter cards disposed in the chassis, the adapter cards comprising hardware and software, the hardware and software for
setting up for simulation of a realistic mix of network traffic on the communications network
simulating the realistic mix of network traffic on the communications network
setting up for engaging in stateful TCP connections with the system under test
engaging in stateful TCP connections with the system under test concurrently with simulating the realistic mix of network traffic on the communications network, resulting in additional network traffic on the communications network
measuring performance of the system under test for the stateful TCP connections under load of the simulated network traffic.
15. The apparatus for testing real-world performance of a system under test of claim 14 wherein the adapter cards include a stateless packet processor for simulating the realistic mix of network traffic on the communications network.
16. The apparatus for testing real-world performance of a system under test of claim 14 further comprising hardware and software for modifying a behavior of the simulated network traffic.
17. The apparatus for testing real-world performance of a system under test of claim 16 , further comprising hardware and software for using performance metrics for the stateful TCP connections to modify the behavior of the simulated network traffic to more closely simulate a realistic mix of network traffic.
18. The apparatus for testing real-world performance of a system under test of claim 17 wherein the performance metrics are selected from the group consisting of retransmission rate, fragmentation, packet sizes, and drop/reset rates.
19. The apparatus for testing real-world performance of a system under test of claim 14 further comprising hardware and software for changing a behavior of the simulated network traffic in response to user instructions.
20. An apparatus for testing real-world performance of a system under test coupled to a communications network, the apparatus comprising:
a chassis; and
one or more adapter cards disposed in the chassis, each adapter card comprising:
a hardware programmable stateless packet processor to simulate a realistic mix of network traffic on the communications network, and
a processor coupled to a memory, the processor executing a TCP stack for
engaging in stateful TCP connections with the system under test concurrently with simulating the realistic mix of network traffic on the communications network, resulting in additional network traffic on the communications network, and
measuring performance metrics of the system under test for the stateful TCP connections under load of the simulated realistic mix of network traffic.
21. The apparatus for testing real-world performance of a system under test of claim 20 , further comprising a controller to instruct the programmable stateless packet processor to modify a behavior of the simulated network traffic.
22. The apparatus for testing real-world performance of a system under test of claim 21 , wherein the controller instructs the programmable stateless packet processor to modify the behavior of the simulated network traffic to more closely simulate a realistic mix of network traffic based on the performance metrics.
23. The apparatus for testing real-world performance of a system under test of claim 22 wherein the performance metrics are selected from the group consisting of retransmission rate, fragmentation, packet sizes, and drop/reset rates.
24. The apparatus for testing real-world performance of a system under test of claim 21 , wherein the controller instructs the programmable stateless packet processor to modify a behavior of the simulated network traffic in response to user instructions.
25. An enterprise load system for testing a system under test available on a communications network, the enterprise load system comprising:
a hardware stateless packet processor for simulating real-world network traffic on the communications network;
a processor coupled to a memory, the processor executing a TCP stack for
generating stateful TCP connections across the communications network with the system under test, each stateful TCP connection including receiving at least one packet from the system under test and sending at least one response packet in response to the received packet, and
measuring performance of the system under test in supporting the stateful TCP connections from the second means in the presence of the simulated traffic on the communication network from the first means; and
a controller coupled to the hardware stateless packet processor, the controller for changing quantity and quality of the network traffic simulated by the hardware stateless packet processor,
wherein the hardware stateless packet processor, the processor generating stateful TCP connections, and the processor measuring performance of the system under test operate concurrently.
26. The enterprise load system for testing a system under test available on a communications network of claim 25 , wherein the system under test comprises a stateful system under test which uses underlying services of TCP.
27. The enterprise load system for testing a system under test available on a communications network of claim 26 , wherein the system under test comprises an HTTP server.
28. The enterprise load system for testing a system under test available on a communications network of claim 26 , wherein the system under test software system comprises an FTP server.
29. The enterprise load system for testing a system under test available on a communications network of claim 25 , the controller further for using performance metrics measured by the processor to cause the hardware stateless packet processor to more closely simulate a realistic mix of network traffic.
30. The enterprise load system for testing a system under test available on a communications network of claim 29 wherein the performance metrics are selected from the group consisting of retransmission rate, fragmentation, packet sizes, and drop/reset rates.
31. The enterprise load system for testing a system under test available on a communications network of claim 25 wherein the controller is responsive to instructions directed to the enterprise load system for changing a behavior of the simulated network traffic.
32. A method of testing a system under test available on a communications network, the method comprising:
simulating real-world network traffic on the communications network
generating stateful TCP connections across the communications network with the system under test
measuring performance metrics of the system under test in supporting the stateful TCP connections in the presence of the simulated real-world network traffic
changing quantity and quality of the simulated real-world network traffic
wherein the steps of simulating, generating and measuring are performed concurrently.
33. The method of testing a system under test available on a communications network of claim 32 , wherein the simulated network traffic is generated by a stateless packet processor.
34. The method of testing a system under test available on a communications network of claim 32 , wherein the system under test comprises a stateful application which uses underlying services of TCP.
35. The method of testing a system under test available on a communications network of claim 34 , wherein the system under test comprises an HTTP server.
36. The method of testing a system under test available on a communications network of claim 34 , wherein the system under test comprises an FTP server.
37. The method of testing a system under test available on a communications network of claim 32 further comprising
modifying a behavior of the simulated network traffic
continuing to generate interactive transactions with the system under test software system
continuing to measure performance of the system under test.
38. The method of testing a system under test available on a communications network of claim 37 , the modifying step comprising using the performance metrics to modify the behavior of the simulated network traffic to more closely simulate a realistic mix of network traffic.
39. The method of testing a system under test of claim 38 wherein the performance metrics are selected from the group consisting of retransmission rate, fragmentation, packet sizes, and drop/reset rates.
40. The method of testing a system under test available on a communications network of claim 37 , the modifying step comprising a user using a control program to change the behavior of the simulated network traffic via a system interface.
41. The method of testing a system under test available on a communications network of claim 40 , the modifying step comprising the user managing multiple ports in a coordinated fashion.