Methods, systems, and computer readable media for vendor-neutral testing and scoring of systems under test

1. A method for vendor-neutral testing and scoring of a system under test, the method comprising:

generating a pretesting snapshot of the system under test, wherein the system under test includes network or data center equipment including at least one server that performs a processing or computing function, the at least one server comprising a file server, a web server, a mapper, a reducer, a search engine, or a social networking server;

executing vendor-neutral testing of plural different subsystems of the system under test, wherein executing vendor-neutral testing of plural different subsystems of the system under test includes executing a virtual machine (VM) test and at least one of: a central processing unit (CPU) performance test, an input/output (I/O), and a big data infrastructure test, wherein executing the VM test includes executing a VM scaling test where the system under test is instructed to instantiate a plurality of VMs on the system under test by ramping a number of VMs on the system under test to a predetermined level and the response time of the system under test to instantiate the virtual machines on the system under test is monitored, wherein the system under test includes a hypervisor and executing the VM scaling test includes:

instructing the hypervisor to serially increase the number VMs to the predetermined level and determining whether there is a degradation in VM instantiation performance over time; and

instructing the hypervisor to increase the number of VMs to the predetermined level in parallel and monitoring the response time includes monitoring a time for the hypervisor to increase the number of VMs to the predetermined level serially and in parallel and wherein the method further comprises:

generating a vendor-neutral score based on results of the testing; and

generating a post-testing snapshot of the system under test.

2. The method of claim 1 wherein generating a pre-testing snapshot of the system under test includes recording hardware, software, and virtual machine configurations of the system under test prior to executing the testing.

3. The method of claim 1 wherein generating a vendor-neutral score based on results of the testing includes assigning point values to each of the tests and generating the score based on the point values.

4. The method of claim 1 wherein executing a VM test includes sending simulated traffic between VMs and monitoring switching of the simulated traffic between the VMs.

5. The method of claim 1 wherein executing an I/O test includes executing an I/O test that spans plural VMs.

6. The method of claim 1 wherein executing an I/O test includes executing at least one of a disk I/O test and a network I/O test.

7. The method of claim 1 wherein executing a big data infrastructure test includes executing a map-reduce test.

8. The method of claim 1 wherein executing the vendor-neutral testing includes testing at least one physical component of the system under test and at least one virtual component of the system under test and wherein generating the vendor-neutral score includes generating a composite score that includes contributions from results of the testing of the at least one physical component and the at least one virtual component.

9. The method of claim 1 comprising repeating the vendor-neutral testing at different times of operation of the system under test, recording the vendor-neutral score at each time, and outputting an indication of how the vendor-neutral scores change over time.

10. A system for vendor-neutral testing and scoring of a system under test, the system comprising:

a network equipment test device including at least one processor;

a tester/scorer implemented by the at least one processor and configured for:

generating a pretesting snapshot of the system under test, wherein the system under test includes network or data center equipment including at least one server that performs a processing or computing function, the at least one server comprising a file server, a web server, a mapper, a reducer, a search engine, or a social networking server;

executing vendor-neutral testing of plural different subsystems of the system under test, wherein executing vendor-neutral testing of plural different subsystems of the system under test includes executing a virtual machine (VM) test and at least one of: a central processing unit (CPU) performance test, an input/output (I/O), and a big data infrastructure test, wherein executing the VM test includes executing a VM scaling test where the system under test is instructed to instantiate a plurality of VMs on the system under test by ramping a number of VMs on the system under test to a predetermined level and the response time of the system under test to instantiate the virtual machines on the system under test is monitored, wherein the system under test includes a hypervisor and executing the VM scaling test includes:

instructing the hypervisor to serially increase the number of VMs to the predetermined level and determining whether there is a degradation in VM instantiation performance over time; and

instructing the hypervisor to increase the number of VMs to the predetermined level in parallel and monitoring the response time includes monitoring a time for the hypervisor to increase the number of VMs to the predetermined level serially and in parallel and wherein the tester/scorer is further configured for:

generating a vendor-neutral score based on results of the testing; and

generating a post testing snapshot of the system under test.

11. The system of claim 10 wherein generating a pre-testing snapshot of the system under test includes recording hardware, software, and virtual machine configurations of the system under test prior to executing the testing.

12. The system of claim 10 wherein generating a vendor-neutral score based on results of the testing includes assigning point values to each of the tests and generating the score based on the point values.

13. The system of claim 10 wherein executing a VM test includes sending simulated traffic between VMs and monitoring switching of the simulated traffic between the VMs.

14. The system of claim 10 wherein executing an I/O test includes executing an I/O test that spans plural VMs.

15. The system of claim 10 wherein executing an I/O test includes executing at least one of a disk I/O test and a network I/O test.

16. The system of claim 10 wherein executing a big data infrastructure test includes executing a map-reduce test.

17. The system of claim 10 wherein the network equipment test device comprises a simulation platform for sending simulated network traffic to the system under test.

18. The system of claim 10 wherein the score includes a virtual machine (VM) certification component indicating successful completion of at least one VM certification test.

19. The system of claim 10 wherein executing the vendor-neutral testing includes testing at least one physical component of the system under test and at least one virtual component of the system under test and wherein generating the vendor-neutral score includes generating a composite score that includes contributions from results of the testing of the at least one physical component and the at least one virtual component.

20. The system of claim 10 comprising repeating the vendor-neutral testing at different times of operation of the system under test, recording the vendor-neutral score at each time, and outputting an indication of how the vendor-neutral scores change over time.

21. A non-transitory computer readable medium having stored thereon executable instructions that when executed by the processor of a computer control the computer to perform steps comprising:

generating a pretesting snapshot of the system under test, wherein the system under test includes network or data center equipment including at least one server that performs a processing or computing function, the at least one server comprising a file server, a web server, a mapper, a reducer, a search engine, or a social networking server;

executing vendor-neutral testing of plural different subsystems of the system under test, wherein executing vendor-neutral testing of plural different subsystems of the system under test includes executing a virtual machine (VM) test and at least one of: a central processing unit (CPU) performance test, an input/output (I/O), and a big data infrastructure test, wherein executing the VM test includes executing a VM scaling test where the system under test is instructed to instantiate a plurality of VMs on the system under test by ramping a number of VMs on the system under test to a predetermined level and the response time of the system under test to instantiate the virtual machines on the system under test is monitored, wherein the system under test includes a hypervisor and executing the VM scaling test includes:

instructing the hypervisor to serially increase the number of VMs to the predetermined level and determining whether there is a degradation in VM instantiation performance over time; and

instructing the hypervisor to increase the number of VMs to the predetermined level in parallel and monitoring the response time includes monitoring a time for the hypervisor to increase the number of VMs to the predetermined level serially and in parallel and wherein the steps further comprise:

generating a vendor-neutral score based on results of the testing; and

generating a post testing snapshot of the system under test.