Method for optimizing test order, and machine-readable media storing sequences of instructions to perform same
View Patent ↗In a test order optimization method, a pass probability and resource requirement are identified for each of a plurality of tests. A pass-weighted resource requirement factor is then generated for each of the tests in accord with the test's pass probability and resource requirement. Thereafter, the tests are ordered to be executed in accord with their pass-weighted resource requirement factors.
1. A test order optimization method, comprising:
identifying a pass probability for each of a plurality of tests;
identifying a resource requirement for each of the tests;
for each of the tests, generating a pass-weighted resource requirement factor in accord with the test's pass probability and resource requirement; and
ordering the tests to be executed in accord with their pass-weighted resource requirement factors.
2. The method of claim 1 , wherein identifying pass probabilities for the tests comprises, for at least one test, estimating a pass probability of the test before the test is executed.
3. The method of claim 1 , wherein identifying pass probabilities for the tests comprises, for at least one test, determining a pass probability of the test from prior executions of the test.
4. The method of claim 1 , wherein identifying resource requirements for the tests comprises, for at least one test, identifying a time requirement necessary to execute the test.
5. The method of claim 1 , wherein generating pass-weighted resource requirement factors for the tests comprises, for at least one test, multiplying the test's pass probability and resource requirement.
6. The method of claim 1 , wherein generating pass-weighted resource requirement factors for the tests comprises, for at least one test, adding the test's pass probability and resource requirement.
7. The method of claim 1 , wherein, for at least one test, the test's pass-weighted resource requirement factor is further generated in accord with an adjustment factor.
8. The method of claim 7 , wherein the adjustment factor for a test reflects the importance of the test.
9. The method of claim 1 , wherein different resource requirements are identified for different tests, and wherein, for at least one test, the test's pass-weighted resource requirement factor is further generated in accord with an adjustment factor that equates its resource requirement with a different resource requirement.
10. The method of claim 1 , wherein generating pass-weighted resource requirement factors for the tests comprises, for at least one test, generating its pass-weighted resource requirement in accord with a non-linear formula.
11. The method of claim 1 , wherein ordering the tests comprises placing tests with lower pass-weighted resource requirement factors for execution before tests with higher pass-weighted resource requirement factors.
12. The method of claim 1 , further comprising, periodically,
updating at least the pass probabilities for the tests;
re-generating the pass-weighted resource requirement factors for the tests; and
re-ordering the tests to be executed in accord with their re-generated pass-weighted resource requirement factors.
13. The method of claim 1 , wherein the tests are circuit tests.
14. A number of machine-readable media having stored thereon sequences of instructions that, when executed by a machine, cause the machine to perform the actions of:
identifying a pass probability for each of a plurality of tests;
identifying a resource requirement for each of the tests;
for each of the tests, generating a pass-weighted resource requirement factor in accord with the test's pass probability and resource requirement; and
ordering the tests to be executed in accord with their pass-weighted resource requirement factors.
15. The machine-readable media of claim 14 , wherein the instructions that cause the machine to perform the action of identifying resource requirements for the tests cause the machine to, for at least one test, identify a time requirement necessary to execute the test.
16. The machine-readable media of claim 14 , wherein the instructions that cause the machine to perform the action of generating pass-weighted resource requirement factors for the tests cause the machine to, for at least one test, multiply the test's pass probability and resource requirement.
17. The machine-readable media of claim 14 , wherein the instructions that cause the machine to perform the action of generating pass-weighted resource requirement factors for the tests cause the machine to, for at least one test, further generate the test's pass-weighted resource requirement factor in accord with an adjustment factor.
18. The machine-readable media of claim 17 , wherein the adjustment factor for a test reflects the importance of the test.
19. The machine-readable media of claim 14 , wherein the instructions further cause the machine to, periodically,
update at least the pass probabilities for the tests;
re-generate the pass-weighted resource requirement factors for the tests; and
re-order the tests to be executed in accord with their re-generated pass-weighted resource requirement factors.