Computer system test parallelization
In accordance with a technique of parallelized computer system testing, a processor develops, on a representative computer system, a plurality of groups of system tests based on test-ordering constraints, where the system tests in each of the plurality of groups are executable in a temporally overlapping manner. The processor applies a test suite including a sequence of multiple of the plurality of groups of system tests to a production computer system. Applying the test suite includes performing the system tests in each of multiple of the plurality of tests groups in a temporally overlapping manner, such that testing efficiency is improved.
1. A method of data processing in a data processing system including a processor, the method comprising:
the processor developing, on a representative computer system, a plurality of groups of system tests based on test-ordering constraints, wherein the system tests in each of the plurality of groups are executable in a temporally overlapping manner;
persistently storing, by the processor, the plurality of groups and a sequence for execution of the groups, the plurality of groups being generated on the representative computer system; and
applying, by the processor, a test suite including a sequence of multiple of the plurality of groups of system tests to a production computer system, wherein the applying comprises executing the system tests in each of multiple of the plurality of groups in a temporally overlapping manner, strictly according to the sequence specified in the plurality of groups.
2. The method of claim 1 , further comprising:
the processor validating the representative computer system through sequential execution of individual system tests forming one of the plurality of groups.
3. The method of claim 1 , wherein the test-ordering constraints include at least one of the following set:
resource constraints; and
data dependency constraints.
4. The method of claim 1 , further comprising:
the processor selecting the test suite for application to the production computer system based on a configuration specified by the production computer system.
5. The method of claim 4 , further comprising:
the processor selecting the sequence of the multiple groups of system tests forming the test suite from among multiple different sequences based at least on a fallout rate for the selected sequence.
6. The method of claim 1 , further comprising:
the processor updating the sequence based on the test suite discovering a defect in the production computer system.
7. The method of claim 1 , wherein selecting the sequence of the plurality of groups of system tests for application to the production computer system comprises weighting selection criteria based on a stage of a product production cycle, the selection criteria including at least one of overall runtime of the test suite, mean fallout rate, and mean runtime to failure.
8. The method of claim 7 , wherein during an initial stage of the product production cycle, the processor selects the sequence of the plurality of groups of system tests by weighting the overall runtime of the test suite more heavily than the mean runtime to failure.
9. A program product, comprising:
a non-transitory computer-readable storage medium; and
program code stored within the non-transitory computer-readable storage medium and executable by processing circuitry of a data processing system to cause the data processing system to perform:
developing, on a representative computer system, a plurality of groups of system tests based on test-ordering constraints, wherein the system tests in each of the plurality of groups are executable in a temporally overlapping manner;
persistently storing the plurality of groups and a sequence for execution of the groups, the plurality of groups being generated on the representative computer system; and
applying a test suite including a sequence of multiple of the plurality of groups of system tests to a production computer system, wherein the applying comprises executing the system tests in each of multiple of the plurality of groups in a temporally overlapping manner, strictly according to the sequence specified in the plurality of groups.
10. The program product of claim 9 , wherein the program code, when executed, further causes the data processing system to perform:
validating the representative computer system through sequential execution of a plurality of individual system tests forming one of the plurality of groups.
11. The program product of claim 9 , wherein the test-ordering constraints include at least one of the following set:
resource constraints; and
data dependency constraints.
12. The program product of claim 9 , wherein the program code, when executed, further causes the data processing system to perform:
selecting the test suite for application to the production computer system based on a configuration specified by the production computer system.
13. The program product of claim 12 , wherein the program code, when executed, further causes the data processing system to perform:
selecting the sequence of the multiple groups of system tests forming the test suite from among multiple different sequences based at least on a fallout rate for the selected sequence.
14. The program product of claim 9 , wherein the program code, when executed, further causes the data processing system to perform:
updating the sequence based on the test suite discovering a defect in the production computer system.
15. A data processing system, comprising:
processing circuitry;
a storage device communicatively coupled to the processing circuitry; and
program code stored within the storage device and executable by the processing circuitry of the data processing system to cause the data processing system to perform:
developing, on a representative computer system, a plurality of groups of system tests based on test-ordering constraints, wherein the system tests in each of the plurality of groups are executable in a temporally overlapping manner;
persistently storing the plurality of groups and a sequence for execution of the groups, the plurality of groups being generated on the representative computer system; and
applying a test suite including a sequence of multiple of the plurality of groups of system tests to a production computer system, wherein the applying comprises executing the system tests in each of multiple of the plurality of groups in a temporally overlapping manner, strictly according to the sequence specified in the plurality of groups.
16. The data processing system of claim 15 , wherein the program code, when executed, further causes the data processing system to perform:
validating the representative computer system through sequential execution of a plurality of individual system tests forming one of the plurality of groups.
17. The data processing system of claim 15 , wherein the test-ordering constraints include at least one of the following set:
resource constraints; and
data dependency constraints.
18. The data processing system of claim 15 , wherein the program code, when executed, further causes the data processing system to perform:
selecting the test suite for application to the production computer system based on a configuration specified by the production computer system.
19. The data processing system of claim 18 , wherein the program code, when executed, further causes the data processing system to perform:
selecting the sequence of the multiple groups of system tests forming the test suite from among multiple different sequences based at least on a fallout rate for the selected sequence.
20. The data processing system of claim 15 , wherein the program code, when executed, further causes the data processing system to perform:
updating the sequence based on the test suite discovering a defect in the production computer system.