Test driven deployment and monitoring of heterogeneous network systems
A test system (and corresponding method and computer program product) for generating unit tests for a heterogeneous network system and validating test results to ensure that the network system functions properly is described. In one embodiment, the test system is an appliance that is capable of normalizing communication protocols supported by component systems of the network system. The test system creates objects and methods corresponding to component systems and their supported protocol commands in the network system, and generates unit test cases based on the objects, the methods, and the normalized protocols. The test system transmits the unit test cases to the component systems, receives test results, and validates the test results to ensure that the network system functions properly.
1. A method for testing a heterogeneous network including a plurality of interconnected component systems, each component system supporting one or more communication protocols, the method comprising:
normalizing communication protocols supported by the plurality of component systems into corresponding protocol grammars;
after the normalizing, representing the plurality of component systems as a set of objects with methods corresponding to supported protocol commands;
after the representing, generating a unit test for the heterogeneous system comprising a collection of method calls that correspond with a series of interactions among the plurality of component systems;
after generating the unit test, converting the collection of method calls into test protocol messages based on the corresponding protocol grammars;
after the converting, transmitting the test protocol messages to the plurality of component systems;
monitoring returning messages received from the plurality of component systems in response to the test protocol messages; and
determining whether the heterogeneous network functions properly based on the returning messages received from the plurality of component systems.
2. The method of claim 1 , wherein the set of objects with methods are displayed in an Integrated Development Environment, and a user can program the collection of method calls using the set of objects and methods.
3. The method of claim 1 , further comprising:
prior to the steps of normalizing and representing, discovering the plurality of component systems and their supported communication protocols and protocol commands by sending messages to component systems and observing responses to those messages.
4. The method of claim 1 , further comprising adding assertions into the returning messages to validate future returning messages.
5. A computer program product having a non-transitory computer-readable medium having computer program instructions recorded thereon for testing a heterogeneous network including a plurality of interconnected component systems, each supporting one or more communication protocols, the computer program instruction comprising instructions for:
normalizing communication protocols supported by the plurality of component systems into corresponding protocol grammars;
representing the plurality of component systems as a set of objects with methods corresponding to supported protocol commands;
generating a unit test for the heterogeneous system comprising a collection of method calls that correspond with a series of interactions among the plurality of component systems;
converting the collection of method calls into test protocol messages based on the corresponding protocol grammars;
transmitting the test protocol messages to the plurality of component systems;
monitoring returning messages received from the plurality of component systems in response to the test protocol messages; and
determining whether the heterogeneous network functions properly based on the returning messages received from the plurality of component systems.
6. The computer program produce of claim 5 , wherein the set of objects with methods are displayed in an Integrated Development Environment, and a user can program the collection of method calls using the set of objects and methods.
7. The computer program product of claim 5 , wherein the computer program instructions further comprise instructions for:
prior the normalizing and representing, discovering the plurality of component systems and their supported communication protocols and protocol commands by sending messages to component systems and observing responses to those messages.
8. The computer program product of claim 5 , wherein the computer program instructions further comprise instructions for:
adding assertions into the returning messages to validate future returning messages.
9. An apparatus for testing a heterogeneous network including a plurality of interconnected component systems, each supporting one or more communication protocols, the apparatus comprising:
a protocol engine configured to normalize communication protocols supported by the plurality of component systems into corresponding protocol grammars, represent the plurality of component systems as a set of objects with methods corresponding to supported protocol commands, generate a unit test for the heterogeneous system comprising a collection of method calls that correspond with a series of interactions among the plurality of component systems, convert the collection of method calls into test protocol messages based on the corresponding protocol grammars, and transmit the test protocol messages to the plurality of component systems;
an assertion engine configured to determine whether the heterogeneous network function properly based on returning messages received from the plurality of component systems in response to the test protocol messages; and
a data store configured to store the protocol grammars, the set of objects with methods, and the collection of method calls.
10. A method for testing a heterogeneous network including a plurality of interconnected component systems, each component system supporting one or more communication protocols, the method comprising:
normalizing communication protocols supported by the plurality of component systems into corresponding protocol grammars;
representing the plurality of component systems as a set of objects with methods corresponding to supported protocol commands;
generating a unit test for the heterogeneous system comprising a collection of method calls that correspond with a series of interactions among the plurality of component systems;
converting the collection of method calls into test protocol messages based on the corresponding protocol grammars;
transmitting the test protocol messages to the plurality of component systems;
monitoring returning messages received from the plurality of component systems in response to the test protocol messages;
determining whether the heterogeneous network functions properly based on the returning messages received from the plurality of component systems;
extracting information regarding interdependencies among the system components;
generating test messages that verify that the interdependencies remain valid;
sending the test messages;
observing the responses to the test messages; and
verifying the interdependencies remain valid among the system components based on the responses to the test messages.
11. A computer program product having a non-transitory computer-readable medium having computer program instructions recorded thereon for testing a heterogeneous network including a plurality of interconnected component systems, each supporting one or more communication protocols, the computer program instruction comprising instructions for:
normalizing communication protocols supported by the plurality of component systems into corresponding protocol grammars;
representing the plurality of component systems as a set of objects with methods corresponding to supported protocol commands;
generating a unit test for the heterogeneous system comprising a collection of method calls that correspond with a series of interactions among the plurality of component systems;
converting the collection of method calls into test protocol messages based on the corresponding protocol grammars;
transmitting the test protocol messages to the plurality of component systems;
monitoring returning messages received from the plurality of component systems in response to the test protocol messages;
determining whether the heterogeneous network functions properly based on the returning messages received from the plurality of component systems;
extracting information regarding interdependencies among the system components;
generating test messages that verify that the interdependencies remain valid;
sending the test messages;
observing the responses to the test messages; and
verifying the interdependencies remain valid among the system components based on the responses to the test messages.
12. An apparatus for testing a heterogeneous network including a plurality of interconnected component systems, each supporting one or more communication protocols, the apparatus comprising:
a protocol engine configured to normalize communication protocols supported by the plurality of component systems into corresponding protocol grammars, represent the plurality of component systems as a set of objects with methods corresponding to supported protocol commands, generate a unit test for the heterogeneous system comprising a collection of method calls that correspond with a series of interactions among the plurality of component systems, convert the collection of method calls into test protocol messages based on the corresponding protocol grammars, and transmit the test protocol messages to the plurality of component systems;
an assertion engine configured to determine whether the heterogeneous network functions properly based on returning messages received from the plurality of component systems in response to the test protocol messages; and
a data store configured to store the protocol grammars, the set of objects with methods, and the collection of method calls,
wherein the protocol engine is further configured to extract information regarding the interdepedencies among the system components, and is configured to generate test messages that verify that the interdependencies remain valid, and to transmit the test messages that verify that the interdependencies remain valid, and wherein the assertion engine is further configured to determine, based on returning responses to the test messages, that the interdependencies remain valid among the system components.