Method, system, and storage medium for estimating and improving test case generation
View Patent ↗A technique for estimating and improving the test coverage for large machines, while accumulating minimum information of past test cases (i.e., minimum feedback) is provided. The technique is scalable in the sense that the number of machine instructions needed to measure the test coverage can range from a few instructions to all the instructions. The technique is easily integrated into existing test generation systems and is applicable to both uni- and multi-processing systems.
1. A method for estimating and improving test case generation, comprising:
classifying a plurality of instructions into a plurality of groups according to architectural behavior;
determining a dominating pool from the groups, the dominating pool being a group having relatively low test coverage in comparison to the other groups;
selecting a condition from a checklist associated with the dominating pool;
randomly selecting an instruction in the dominating pool; and
generating test cases for the selected instruction according to the condition.
2. The method of claim 1 , further comprising:
biasing test data for the test cases to improve test coverage.
3. The method of claim 2 , wherein the test coverage includes test input and output conditions.
4. The method of claim 1 , further comprising:
selecting at least one group agent for each group;
wherein the selected instruction is the group agent for the dominating pool.
5. The method of claim 4 , further comprising estimating test coverage for the group with test coverage for the group agent.
6. The method of claim 1 , wherein test cases are generated pseudo-randomly.
7. A system for estimating and improving test case generation, comprising:
a storage medium storing checklist information, the checklist information including test conditions for a plurality of instructions classified into a plurality of groups according to architectural behavior;
a bias scheme component for determining a dominating pool from the groups, the dominating pool being a group having relatively low test coverage in comparison to the other groups, selecting a condition from a checklist associated with the dominating pool, and randomly selecting an instruction in the dominating pool; and
a test case build component for generating test cases for the selected instruction according to the condition.
8. The system of claim 7 , further comprising:
a simulation component for running the test cases and for updating checklist for each of a plurality of group agents selected for each group, wherein the selected instruction is the group agent for the dominating pool.
9. The system of claim 7 , further comprising:
a checklist update component for updating test coverage information for the group based on test cases and test results.
10. The system of claim 7 , further comprising:
a second storage medium for storing test coverage and history information.
11. A computer program product, comprising:
a computer storage medium including computer readable computer program code for estimating and improving test case generation; and
instructions for causing a computer to implement a method, the method further comprising:
classifying a plurality of instructions into a plurality of groups according to architectural behavior;
determining a dominating pool from the groups, the dominating pool being a group having relatively low test coverage in comparison to the other groups;
selecting a condition from a checklist associated with the dominating pool;
randomly selecting an instruction in the dominating pool; and
generating test cases for the selected instruction according to the condition.
12. The computer program product of claim 11 , wherein the method further comprises:
biasing test data for the test cases to improve test coverage.
13. The computer program product of claim 11 , wherein the method further comprises:
selecting at least one group agent for each group;
wherein the selected instruction is the group agent for the dominating pool.
14. The computer program product of claim 13 , wherein the method further comprises:
estimating test coverage for the group with test coverage for the group agent.