Test case selection and ordering with covert minimum set cover for functional qualification
Techniques and systems for test case selection and ordering with covert minimum set cover for functional qualification are described. Some embodiments can determine a first set of test cases by, iteratively, identifying a set of faults that is covered by a smallest set of test cases, determining whether or not a test case that covers a fault is able to detect the fault, and selecting and adding a test case to the first set of test cases. Next, the embodiments can execute a minimum set cover process on the first set of test cases by using coverage scores for test cases in the first set of test cases for ranking.
1. A non-transitory computer-readable storage medium storing instructions, which when executed by a processor, cause the processor to:
receive a first set of faults and a first set of test cases, wherein each fault in the first set of faults is associated with a defect in a circuit design, and wherein each fault in the first set of faults is covered by one or more test cases in the first set of test cases;
determine a least covered fault in the first set of faults, wherein a total number of test cases in the first set of test cases which covers the least covered fault is a lowest total number of test cases compared to respective total number of test cases covering each fault in the first set of faults;
determine whether a first test case in the first set of test cases which covers the least covered fault detects the least covered fault when the least covered fault is injected into a simulation of the circuit design; and
remove faults from the first set of faults which are covered by the first test case when the first test case detects the least covered fault.
2. The non-transitory computer-readable storage medium of claim 1 , wherein faults are removed from the first set of faults iteratively, and wherein a termination condition is met when a predetermined fraction of all fault and test case pairs have been processed.
3. The non-transitory computer-readable storage medium of claim 2 , wherein the predetermined fraction is less than 0.1.
4. The non-transitory computer-readable storage medium of claim 1 , wherein the first test case covers a largest number of faults among test cases in the first set of test cases which cover the least covered fault.
5. The non-transitory computer-readable storage medium of claim 1 , wherein the instructions, which when executed by the processor, cause the processor to remove the first test case from the first set of test cases.
6. The non-transitory computer-readable storage medium of claim 1 , wherein faults in the first set of faults which are covered by the first test case are not removed when the first test case does not detect the least covered fault.
7. The non-transitory computer-readable storage medium of claim 1 , wherein tests are removed from the first set of tests iteratively, and wherein a termination condition is met when the first set of tests is empty.
8. An apparatus, comprising:
a processor; and
a non-transitory computer-readable storage medium storing instructions, which when executed by the processor, cause the processor to:
receive a first set of faults and a first set of test cases, wherein each fault in the first set of faults is associated with a defect in a circuit design, and wherein each fault in the first set of faults is covered by one or more test cases in the first set of test cases;
determine a least covered fault in the first set of faults, wherein a total number of test cases in the first set of test cases which covers the least covered fault is a lowest total number of test cases compared to respective total number of test cases covering each fault in the first set of faults:
determine whether a first test case in the first set of test cases which covers the least covered fault detects the least covered fault when the least covered fault is injected into a simulation of the circuit design; and
remove faults from the first set of faults which are covered by the first test case when the first test case detects the least covered fault.
9. The apparatus of claim 8 , wherein faults are removed from the first set of faults iteratively, and wherein a termination condition is met when a predetermined fraction of all fault and test case pairs have been processed.
10. The apparatus of claim 9 , wherein the predetermined fraction is less than 0.1.
11. The apparatus of claim 8 , wherein the first test case covers a largest number of faults among test cases in the first set of test cases which cover the least covered fault.
12. The apparatus of claim 8 , wherein the instructions, which when executed by the processor, cause the processor to remove the first test case from the first set of test cases.
13. The apparatus of claim 8 , wherein faults from the first set of faults which are covered by the first test case are not removed when the first test case does not detect the least covered fault.
14. The apparatus of claim 8 , wherein tests are removed from the first set of tests iteratively, and wherein a termination condition is met when the first set of tests is empty.
15. A method for test case selection and ordering with covert minimum set cover for functional qualification, the method comprising:
receiving a first set of faults and a first set of test cases, wherein each fault in the first set of faults is associated with a defect in a circuit design, and wherein each fault in the first set of faults is covered by one or more testcases in the first set of test cases;
determining a least covered fault in the first set of faults, wherein a total number of test cases in the first set of test cases which covers the least covered fault is a lowest total number of test cases compared to respective total number of test cases covering each fault in the first set of faults:
determining whether a first test case in the first set of test cases which covers the least covered fault detects the least covered fault when the least covered fault is injected into a simulation of the circuit design; and removing, by a processor, faults from the first set of faults which are covered by the first test case when the first test case detects the least covered fault, wherein faults from the first set of faults which are covered by the first testcase are not removed when the first test case does not detect the least covered fault.
16. The method of claim 15 , wherein faults are removed from the first set of faults iteratively, and wherein a termination condition is met when a predetermined fraction of all fault and test case pairs have been processed.
17. The method of claim 16 , wherein the predetermined fraction is less than 0.1.
18. The method of claim 15 , wherein the first test case covers a largest number of faults among test cases in the first set of test cases which cover the least covered fault.
19. The method of claim 15 , further comprising removing the first test case from the first set of test cases.
20. The method of claim 15 , wherein tests are removed from the first set of tests iteratively, and wherein a termination condition is met when the first set of tests is empty.