Implicit coordination of deployment and regression testing across data centers and system clusters
The disclosed methods and systems for coordinating deployment and regression testing across data centers and system clusters in multiple service clouds include accessing test suites via a versioning platform that maintains a metadata notation of an applicable-version to which a version of a test suite applies. They include receiving a request-to-test to run a selected test suite against a selected system; making a machine-to-machine query about the selected system and receiving back a response that specifies a version-to-be-tested; automatically selecting from the versioning platform a test suite version with the applicable-version that supports the version-to-be-tested consistent with the metadata notation of the applicable-version of the selected test suite; and reporting results of running the selected test suite. The request-to-test need not include the version of the selected test to run, and can specify use of authentication credentials and multiple selected test suites to run against one or more systems.
1. A method of coordinating deployment and regression testing across data centers and system clusters, including:
accessing test suites via a versioning platform that maintains a metadata notation of an applicable-version of a product to which a particular version of a test suite applies;
receiving a request-to-test from a device or user that includes a request to run a selected test suite against a selected system;
making a machine-to-machine query for information about the selected system and receiving back a response that specifies a product version-to-be-tested;
automatically selecting from the versioning platform a most recent test suite version that is associated with the product version-to-be-tested, consistent with the metadata notation of the applicable-version of the selected test suite version, by disqualifying test suite versions associated with only more recent product versions than the version-to-be-tested;
selecting a client account registered with the selected system for the product;
obtaining authentication credentials for the selected client account to use with the selected system when running the selected test suite;
injecting the selected client account into the selected test suite at runtime;
running the selected test suite version of the selected test suite against the selected system with authentication credentials for the selected client account; and
reporting results of running the selected test suite.
2. The method of claim 1 , further including managing associations of an applicable-version needed to run, with the particular version of the test suite.
3. The method of claim 1 , wherein the request-to-test is silent regarding a version of the selected test suite to run against a selected system and relies on the automatically selecting.
4. The method of claim 3 , wherein the request-to-test specifies multiple selected test suites to run against a selected system.
5. The method of claim 4 , wherein the request-to-test specifies multiple systems against which to run the multiple selected test suites.
6. The method of claim 5 , further comprising, for each system of the multiple systems:
automatically selecting, from the versioning platform, a most recent test suite version that is associated, in the metadata notation, with a product version-to-be-tested present on the each system,
wherein, for at least two of the multiple systems having different versions-to-be-tested of the same product, the same test suite version is selected.
7. The method of claim 1 , further including making the machine-to-machine query directly to the selected system.
8. The method of claim 1 , further including making the machine-to-machine query to a version tracking repository that stores applicable test suite versions.
9. The method of claim 1 , wherein the version-to-be-tested is one of a version of a product release, a hot fix for a version of a product between product releases, or an extended version of a product release.
10. A system that coordinates deployment and regression testing to multiple service clouds spanning multiple data centers that include one or more hardware stacks per data center, the system including deployment and regression testing controllers, including:
the deployment and regression testing controllers deployed at each of the service clouds, each including a processor, memory coupled to the processor, and program instructions executable on the processor;
wherein the program instructions, when executed, cause the respective deployment and regression testing controller to implement a method including:
accessing test suites via a versioning platform that maintains a metadata notation of an applicable-version of a product to which a particular version of a test suite applies;
receiving a request-to-test from a device or user that includes a request to run a selected test suite against a selected system;
making a machine-to-machine query for information about the selected system and receiving back a response that specifies a product version-to-be-tested;
automatically selecting from the versioning platform a most recent test suite version that is associated with the product version-to-be-tested, consistent with the metadata notation of the applicable-version of the selected test suite version, by disqualifying the test suite versions associated with only more recent product versions than the version-to-be-tested;
selecting a client account registered with the selected system for the product;
obtaining authentication credentials for the selected client account to use with the selected system when running the selected test suite;
injecting the selected client account into the selected test suite at runtime;
running the selected test suite version of the selected test suite against the selected system with authentication credentials for the selected client account; and
reporting results of running the selected test suite.
11. The system of claim 10 , wherein the program instructions, when executed, further implement managing associations of an applicable-version needed to run, with the particular version of the test suite.
12. The system of claim 10 , wherein the request-to-test is silent regarding a version of the selected test suite to run against a selected system and relies on the automatically selecting.
13. The system of claim 12 , wherein the request-to-test specifies multiple selected test suites to run against a selected system.
14. The system of claim 13 , wherein the request-to-test specifies multiple systems against which to run the multiple selected test suites.
15. The system of claim 10 , wherein the program instructions, when executed, further implement making the machine-to-machine query to the selected system.
16. The system of claim 10 , wherein the program instructions, when executed, further implement making the machine-to-machine query to a version tracking repository that stores applicable test suite versions.
17. The system of claim 10 , wherein the version-to-be-tested is one of a version of a product release, a hot fix for a version of a product between product releases, or an extended version of a product release.
18. A non-transitory computer readable medium, including program instructions executable on a processor, wherein the instructions, when executed, implement: at deployment and regression testing controllers at each service cloud, each including a processor, a method including:
accessing test suites via a versioning platform that maintains a metadata notation of an applicable-version of a product to which a particular version of a test suite applies;
receiving a request-to-test from a device or user that includes a request to run a selected test suite against a selected system;
making a machine-to-machine query for information about the selected system and receiving back a response that specifies a product version-to-be-tested;
automatically selecting from the versioning platform a most recent test suite version that is associated with the product version-to-be-tested, consistent with the metadata notation of the applicable-version of the selected test suite version, by disqualifying the test suite versions associated with only more recent product versions than the version-to-be-tested;
selecting a client account registered with the selected system for the product;
obtaining authentication credentials for the selected client account to use with the selected system when running the selected test suite;
injecting the selected client account into the selected test suite at runtime;
running the selected test suite version of the selected test suite against the selected system with authentication credentials for the selected client account; and
reporting results of running the selected test suite.
19. The non-transitory computer readable medium of claim 18 , further including managing associations of an applicable-version needed to run, with the particular version of the test suite.
20. The non-transitory computer readable medium of claim 18 , wherein the request-to-test specifies multiple systems against which to run selected test suites.
21. The non-transitory computer readable medium of claim 18 , further including making the machine-to-machine query to a version tracking repository that stores applicable test suite versions.
22. The non-transitory computer readable medium of claim 18 , wherein the version-to-be-tested is one of a version of a product release, a hot fix for a version of a product between product releases, or an extended version of a product release.