Systems and methods for continuous integration automated testing in a distributed computing environment

1. A method for continuous integration automated testing in a distributed computing environment, comprising:

in an information processing device comprising at least one computer processor:

receiving computer-executable code for a computer application into an automated testing environment;

using a plurality of testbed servers, executing a baseline system test run on the computer-executable code for the computer application by testing the computer-executable code with a set of input test data that is optimized for test execution time and test coverage and storing baseline test run results;

receiving new computer-executable code for the computer application at a test environment;

determining that the new computer-executable code for the computer application differs from the computer-executable code for the computer application;

using the plurality of testbed servers, executing an system test run on the new computer-executable code for the computer application by testing the new computer-executable code with the set of input test data in the distributed computing environment and storing test run results;

comparing the test run results to the baseline test run results;

generating at least one report based on the comparing;

identifying a regression in the test run results;

automatically identifying a commit that caused the regression;

disabling the commit that caused the regression; and

iteratively disabling commits and repeating the test run until a change that caused the regression is identified.

2. The method of claim 1 , wherein the computer-executable code for the computer application is loaded from a production database for the computer application.

3. The method of claim 2 , wherein the computer-executable code for the computer application comprises a full copy of the production database for the computer application.

4. The method of claim 2 , wherein the computer-executable code for the computer application comprises a portion of the production database for the computer application.

5. The method of claim 2 , wherein the baseline run is executed automatically in response to a change in the production database for the computer application.

6. The method of claim 1 , wherein the baseline run is executed automatically in response to a change in the computer application.

7. The method of claim 6 , wherein the change comprises at least a change in an output of the computer application, a change in compute resources for the computer application, and a change in performance of the computer application.

8. The method of claim 1 , wherein the test environment receives the new computer-executable code for the computer application from a master branch of a version control system.

9. The method of claim 1 , wherein the test environment receives the new computer-executable code for the computer application from an alternate branch of a version control system.

10. The method of claim 1 , wherein the test run is an on-demand test run.

11. The method of claim 1 , wherein the test run is a scheduled test run.

12. A continuous integration automated testing system, comprising:

a production database for a computer application;

a plurality of testbed servers; and

a server comprising at least one computer processor and a memory, wherein the at least one computer processor is configured to perform the following:

receive computer-executable code for the computer application from the production database;

using the plurality of testbed servers, execute a baseline system test run on the computer-executable code for the computer application by testing the computer-executable code with a set of input test data that is optimized for test execution time and test coverage;

store the baseline system test run results;

receive new computer-executable code for the computer application;

determine that the new computer-executable code for the computer application differs from the computer-executable code for the computer application;

using the plurality of testbed servers, execute an system test run on the new computer-executable code for the computer application by testing the new computer-executable code with the set of input test;

store the test run results;

compare the test run results to the baseline test run results;

generate at least one report based on the comparison;

identify a regression in the test run results;

automatically identify a commit that caused the regression;

disable the commit that caused the regression; and

iteratively disable commits and repeat the test run until a change that caused the regression is identified.

13. The system of claim 12 , wherein the computer-executable code for the computer application is loaded from a production database for the computer application.

14. The system of claim 13 , wherein the computer-executable code for the computer application comprises a full copy of the production database for the computer application.

15. The system of claim 13 , wherein the computer-executable code for the computer application comprises a portion of the production database for the computer application.

16. The system of claim 13 , wherein the baseline run is executed automatically in response to a change in the production database for the computer application.

17. The system of claim 12 , wherein the baseline run is executed automatically in response to a change in the computer application.

18. The system of claim 17 , wherein the change comprises at least a change in an output of the computer application, a change in compute resources for the computer application, and a change in performance of the computer application.

19. The system of claim 12 , wherein the new computer-executable code for the computer application is received from a master branch of a version control system.

20. The system of claim 12 , wherein the new computer-executable code for the computer application is received from an alternate branch of a version control system.