Multi-component application test framework
A multi-component application test framework tests components of an application out of order and without all components being ready for test. Descriptions for each component of an application are received and stored. Definitions for waypoints (code or test data substitutes for actual components net yet submitted) are received and stored. An endpoint lookup directory is initialized with waypoint endpoints for each component. A particular component is submitted for test and the test deployed. The particular component dynamically looks up endpoints for input/output dependencies for the particular component with other components and activates inactive waypoints acting as substitutes for components not-yet-submitted. The test completes, and for a passing test, the endpoint directory is updated to point to the particular component for other component dependencies instead of waypoints, effectively merging the particular component with others dependent on it. A similar process is repeated until all components are merged.
1 . A system, comprising:
one or more computers comprising respective processors and memory configured to implement a multi-component application test framework configured to:
receive a test description for a multi-component application to be tested, the description identifying respective components of a plurality of components of the multi-component application and identifying dependencies between the plurality of components;
determine, based on the identified dependencies between the plurality of components, respective input and output waypoints for individual components of the plurality of components, wherein each input waypoints is configured to simulate one of the components that provides input to one or more other ones of the components, and each output waypoint is configured to simulate one of the components that receives output from one or more other ones of the components;
set endpoint information for components that have not been successfully tested to point to corresponding waypoints;
receive and deploy a particular component of the plurality of components for test, wherein the particular component looks up endpoint information for its component dependencies, wherein the endpoint information points to either waypoints or to successfully tested ones of the plurality of components; and
responsive to successful testing of the particular component:
update the endpoint information for ones of the plurality of components dependent on the particular component to point to the particular component instead of an input or output waypoint corresponding to the particular component;
determine whether any active waypoints remain for dependencies between the plurality of components;
continue application testing if any active waypoints remain for dependencies between the plurality of components; and
report successful completion of application testing if no active waypoints remain for dependencies between the plurality of components.
2 . The system of claim 1 , wherein:
the multi-component application test framework is configured to receive one or more different ones of the components for testing at different times, and
each component is tested using other components or waypoints for its dependencies according to the endpoint information when the component is submitted for testing.
3 . The system of claim 1 , wherein the multi-component application test framework is configured to, responsive to successful testing of the particular component, deactivate waypoints corresponding the particular component.
4 . The system of claim 1 , wherein the multi-component application test framework is configured to:
subsequent to completion of the test:
receive one or more components, that were simulated in the test, for a second test, wherein at least one or more others of the components that are not yet received for the second test are simulated by a waypoint for the second test; and
execute the second test such that each of the received one or more components is tested using other components or waypoints for respective dependencies according to the endpoint information when the received one or more components was submitted for testing via the second test.
5 . The system of claim 1 , wherein:
said set endpoint information for components that have not been successfully tested to point to corresponding waypoints comprises set the endpoint information in an endpoint directory data store;
said update endpoint information to point to the particular component instead of an input or output waypoint corresponding to the particular component comprises update the endpoint information in the endpoint directory data store; and
the multi-component application test framework is configured to:
disseminate endpoint information from the endpoint directory data store to one or more component registries.
6 . A method, the method comprising:
determining, by a multi-component application test framework implemented by one or more processors and based on dependencies, specified by a test description, between a plurality of components of a multi-component application to be tested, respective input and output waypoints for individual components of the plurality of components, wherein each input waypoint is configured to simulate one of the components that provides input to one or more other ones of the components, and each output waypoint is configured to simulate one of the components that receives output from one or more other ones of the components;
setting endpoint information, in an endpoint directory data store, for components that have not been successfully tested to point to corresponding waypoints;
deploying a particular component of the plurality of components for test, wherein the particular component looks up endpoint information for endpoints that point to either waypoints or to successfully tested ones of the plurality of components, for the particular component dependencies;
updating, responsive to successful testing of the particular component, the endpoint information, in the endpoint directory data store, for ones of the plurality of components dependent on the particular component to point to the particular component instead of an input or output waypoint corresponding to the particular component; and
disseminating endpoint information from the endpoint directory data store to one or more component registries.
7 . The method of claim 6 , further comprising:
subsequent to completion of the test:
receiving one or more components, that were simulated in the test, for a second test, wherein at least one or more others of the components that are not yet received for the second test are simulated by a waypoint for the second test; and
executing the second test such that each of the received one or more components is tested using other components or waypoints for respective dependencies according to the endpoint information when the received one or more components was submitted for testing via the second test.
8 . The method of claim 6 , further comprising generating, responsive to unsuccessful testing of the particular component, a report of test failures and leave waypoints corresponding to the particular component in place so that the ones of the plurality of components dependent on the particular component will interact with respective ones of the waypoints corresponding to the particular component for testing.
9 . The method of claim 8 , further comprising:
obtaining, during runtime and by the particular component from the one or more component registries, the endpoint information for endpoints that point to either waypoints or to successfully tested ones of the plurality of components; and
invoking, during runtime and by the particular component, the obtained endpoint information.
10 . The method of claim 6 , wherein to simulate one of the components that provides input to one or more other ones of the components, a respective input waypoint is configured to perform:
synthesizing test data to provide as input to the one or more other components; or
replaying captured production data as input to the one or more other components.
11 . The method of claim 6 , further comprising verifying, by respective ones of the output waypoints, the output from one or more other ones of the components according to test criteria.
12 . The method of claim 6 , further comprising:
performing registration and testing of another multi-component application at least partially in parallel with the registration and testing of the multi-component application, wherein performing testing comprises:
determining active waypoints remain for dependencies between a plurality of components of the other multi-component application; and
continuing, based on determining active waypoints remain for dependencies between a plurality of components of the other multi-component application, application testing of the remaining active waypoints.
13 . One or more non-transitory computer-readable media storing program instructions executable on or across one or more processors to implement a multi-component application test framework configured to perform:
determining, by a multi-component application test framework and based on dependencies, specified by a test description, between a plurality of components of a multi-component application to be tested, respective input and output waypoints for individual components of the plurality of components, wherein each input waypoint is configured to simulate one of the components that provides input to one or more other ones of the components, and each output waypoint is configured to simulate one of the components that receives output from one or more other ones of the components;
setting endpoint information for components that have not been successfully tested to point to corresponding waypoints;
deploying a particular component of the plurality of components for test, wherein the particular component looks up endpoint information for endpoints, that points to either waypoints or to successfully tested ones of the plurality of components, for the particular component dependencies;
responsive to successful testing of the particular component:
updating the endpoint information for ones of the plurality of components dependent on the particular component to point to the particular component instead of an input or output waypoint corresponding to the particular component;
determining whether any active waypoints remain for dependencies between the plurality of components;
continuing application testing if any active waypoints remain for dependencies between the plurality of components; and
reporting successful completion of application testing if no active waypoints remain for dependencies between the plurality of components.
14 . The one or more non-transitory computer-readable media of claim 13 , wherein the program instructions cause the multi-component application test framework to perform:
receiving one or more different ones of the components for testing at different times; and
testing each of the received one or more different components using other ones of the components or waypoints for respective component dependencies according to the endpoint information when the one or more different components is submitted for testing.
15 . The one or more non-transitory computer-readable media of claim 13 , wherein the program instructions cause the multi-component application test framework to perform, deactivating, responsive to successful testing of the particular component, waypoints corresponding the particular component.
16 . The one or more non-transitory computer-readable media of claim 13 , wherein:
the program instructions cause the multi-component application test framework to perform receiving component descriptions for the plurality of components; and
setting endpoint information for components that have not been successfully tested comprises initializing an endpoint directory with waypoints for each untested component and propagating the endpoint information to other lookup systems per the component descriptions.
17 . The one or more non-transitory computer-readable media of claim 13 , wherein the program instructions cause the multi-component application test framework to perform:
initiating the test of the particular component; and
responsive to the particular component passing the test,
updating, in an application component status data store of the multi-component application test framework, a status for the particular component to indicate the particular component passed the test, wherein status of one or more others of the components of the multi-component application remain set to a planned, a failed, or a released status.
18 . The one or more non-transitory computer-readable media of claim 13 , wherein the program instructions cause the multi-component application test framework to perform:
obtaining, during runtime and by the particular component from one or more component registries, endpoint information for the endpoints that point to either waypoints or to successfully tested ones of the plurality of components; and
invoking, during runtime and by the particular component, the obtained endpoint information.
19 . The one or more non-transitory computer-readable media of claim 13 , wherein the program instructions cause the multi-component application test framework to perform:
determining, based on an analysis of elimination of the waypoints of the multi-component application, whether all components of the multi-component application have been merged; and
reporting, based on a determination that all components of the multi-component application have been merged, overall application integration test complete.