Deployment testing for infrastructure delivery automation
The disclosed deployment testing system includes a test loader that loads and parses test objects and the antecedent test objects on which the selected test object depends, to prepare a test sequence. A database of test objects apply to one or more SUT and specify aspects of deployed SUT configuration that include hardware configuration, active connection of the SUT to other network devices, configuration of the actively connected network devices to accept access by the SUT, presence of files and applications in storage, and services running on the SUT; identify dependencies on antecedent test objects; specify test object features that extend object features found in the antecedent test objects; and override test parameter values in the antecedent test objects. A test executor obtains administrative credentials for accessing and testing a SUT and uses the test and antecedent test objects to verify the SUT; and a test reporter publishes tester results.
1. A deployment testing system coupled to a computer network, the deployment testing system comprising:
a memory, and
at least one processor coupled to the memory, wherein the at least one processor is configured to:
obtain administrative credentials to access and test at least one system under test (SUT);
load a selected test object and antecedent test objects on which the selected test object depends from a test object database of a plurality of sets of test objects and respective antecedent test objects,
wherein the plurality of sets of test objects and respective antecedent test objects comprise configuration parameters of the at least one SUT, and
wherein the configuration parameters are configured to validate the at least one SUT;
parse the selected test object and the antecedent test objects that are loaded;
build a test sequence using the selected test object and the antecedent test objects as parsed;
wherein to build the test sequence, the at least one processor is further configured to:
identify dependencies on the parsed antecedent test objects, including nested dependencies; and
extend first object features of the parsed selected test object with second object features of the parsed antecedent test objects, comprising:
overriding at least one first object feature of the first object features with test parameter values from the second object features; and
inheriting the dependencies on the parsed antecedent test objects, including the nested dependencies, into the first object features as additional test parameter values;
run the test sequence to verify the configuration parameters of the at least one SUT using the extended first object features having the test parameter values and the additional test parameter values; and
provide results of the run of the test sequence.
2. The deployment testing system of claim 1 , wherein the at least one processor is further configured to:
identify, using the plurality of sets of test objects and respective antecedent test objects, a target SUT of the at least one SUT by at least one service role performed by the target SUT; and
verify the at least one service role of the target SUT.
3. The deployment testing system of claim 1 , wherein the at least one processor is further configured to:
provide the results of the run of the test sequence to a watcher service on a client device.
4. The deployment testing system of claim 1 , wherein the at least one processor is further configured to:
identify, using the plurality of sets of test objects and respective antecedent test objects, a cluster of systems under test of the at least one SUT;
and
verify the identified cluster of systems under test.
5. The deployment testing system of claim 1 , wherein the at least one processor is further configured to:
identify, using the plurality of sets of test objects and respective antecedent test objects, a data center; and
verify the at least one SUT in the identified data center.
6. The deployment testing system of claim 1 , wherein the configuration parameters comprise data specifying a plurality of hardware configurations of the at least one SUT, a plurality of active connections of the at least one SUT to other network devices, a plurality of configurations of actively connected network devices to accept access by the at least one SUT, identification of files and applications in storage of the at least one SUT, identification of services installed on the at least one SUT, or a plurality of configurations of service initialization parameters for the services installed on the at least one SUT.
7. The deployment testing system of claim 1 , wherein the at least one processor is further configured to:
communicate with a subscriber service to access the results of the run of the test sequence.
8. A method of deployment testing of at least one system under test (SUT) of a computer network, the method comprising:
obtaining administrative credentials from an authorization service to access and test the at least one SUT;
loading a selected test object and antecedent test objects on which the selected test object depends from a test object database of a plurality of sets of test objects and respective antecedent test objects,
wherein the plurality of sets of test objects and respective antecedent test objects comprise configuration parameters of the at least one SUT, and
wherein the configuration parameters are configured to validate the at least one SUT;
parsing the selected test object and the antecedent test objects that are loaded;
building a test sequence using the selected test object and the antecedent test objects as parsed;
wherein building the test sequence further comprises:
identifying dependencies on the parsed antecedent test objects, including nested dependencies; and
extending first object features of the parsed selected test object with second object features of the parsed antecedent test objects, wherein the extending comprises:
overriding at least one first object feature of the first object features with test parameter values from the second object features; and
inheriting the dependencies on the parsed antecedent test objects, including the nested dependencies, into the first object features as additional test parameter values;
running the test sequence to verify the configuration parameters of the at least one SUT using the extended first object features having the test parameter values and the additional test parameter values; and
providing results of the run of the test sequence.
9. The method of claim 8 , further comprising:
identifying, using the plurality of sets of test objects and respective antecedent test objects, a target SUT of the at least one SUT by at least one service role performed by the target SUT; and
verifying the at least one service role of the target SUT.
10. The method of claim 8 , further comprising:
identifying, using the plurality of sets of test objects and respective antecedent test objects, a cluster of systems under test of the at least one SUT; and
verifying the identified cluster of systems under test.
11. The method of claim 8 , further comprising:
identifying, using the plurality of sets of test objects and respective antecedent test objects, a data center; and
verifying the at least one SUT in the identified data center.
12. The method of claim 8 , wherein the configuration parameters comprise data specifying a plurality of hardware configurations of the at least one SUT, a plurality of active connections of the at least one SUT to other network devices, a plurality of configurations of actively connected network devices to accept access by the at least one SUT, identification of files and applications in storage of the at least one SUT, identification of services installed on the at least one SUT, or a plurality of configurations of service initialization parameters for the services installed on the at least one SUT.
13. The method of claim 8 , further comprising:
communicating with a subscriber service to access the results of the run of the test sequence.
14. The method of claim 8 , further comprising:
providing the results of the run of the test sequence to a watcher service on a client device.
15. A tangible non-transitory computer-readable device having instructions stored thereon that, when executed by a computing device, cause the computing device to perform operations comprising:
obtaining administrative credentials from an authorization service to access and test at least one system under test (SUT) of a computer network;
loading a selected test object and antecedent test objects on which the selected test object depends from a test object database of a plurality of sets of test objects and respective antecedent test objects,
wherein the plurality of sets of test objects and respective antecedent test objects comprise configuration parameters of the at least one SUT, and
wherein the configuration parameters are configured to validate the at least one SUT;
parsing the selected test object and the antecedent test objects that are loaded;
building a test sequence using the selected test object and the antecedent test objects as parsed;
wherein building the test sequence further comprises:
identifying dependencies on the parsed antecedent test objects, including nested dependencies; and
extending first object features of the parsed selected test object with second object features of the parsed antecedent test objects, wherein the extending comprises:
overriding at least one first object feature of the first object features with test parameter values from the second object features; and
inheriting the dependencies on the parsed antecedent test objects, including the nested dependencies, into the first object features as additional test parameter values;
running the test sequence to verify the configuration parameters of the at least one SUT using the extended first object features having the test parameter values and the additional test parameter values; and
providing results of the run of the test sequence.
16. The computer-readable device of claim 15 , the operations performed by the computing device further comprising:
identifying, using the plurality of sets of test objects and respective antecedent test objects, a target SUT of the at least one SUT by at least one service role performed by the target SUT; and
verifying the at least one service role of the target SUT.
17. The computer-readable device of claim 15 , the operations performed by the computing device further comprising:
identifying, using the plurality of sets of test objects and respective antecedent test objects, a cluster of systems under test of the at least one SUT; and
verifying the identified cluster of systems under test.
18. The computer-readable device of claim 15 , the operations performed by the computing device further comprising:
identifying, using the plurality of sets of test objects and respective antecedent test objects, a data center; and
verifying the at least one SUT in the identified data center.
19. The computer-readable device of claim 15 , wherein the configuration parameters comprise data specifying a plurality of hardware configurations of the at least one SUT, a plurality of active connections of the at least one SUT to other network devices, a plurality of configurations of actively connected network devices to accept access by the at least one SUT, identification of files and applications in storage of the at least one SUT, identification of services installed on the at least one SUT, or a plurality of configurations of service initialization parameters for the services installed on the at least one SUT.
20. The computer-readable device of claim 15 , the operations performed by the computing device further comprising:
communicating with a subscriber service to access the results of the run of the test sequence.
21. The computer-readable device of claim 15 , the operations performed by the computing device further comprising:
providing the results of the run of the test sequence to a watcher service on a client device.