Machine-learning-based integrated circuit test case selection
Certain aspects of the present disclosure provide techniques and apparatus for testing integrated circuit designs. An example method generally includes generating a coverage matrix associated with a plurality of test cases for an integrated circuit and coverage points associated with each test case of the plurality of test cases. A subset of the plurality of test cases is selected for execution based on weights associated with each test case of the plurality of test cases and a threshold weight value. Generally, the weights associated with each test case comprise weights in a machine learning model trained based on the coverage matrix. The integrated circuit may be tested based on the selected subset of test cases.
1 . A processor-implemented method, comprising:
generating a coverage matrix associated with a plurality of test cases for an integrated circuit and coverage points in a universe of coverage points associated with each test case of the plurality of test cases;
selecting a subset of the plurality of test cases for execution on the integrated circuit based on weights associated with each test case of the plurality of test cases and a threshold weight value, wherein the weights associated with each test case of the plurality of test cases comprise weights in a machine learning model trained based on the coverage matrix; and
testing the integrated circuit based on the selected subset of test cases;
wherein selecting the subset of the plurality of test cases for execution comprises selecting a minimal number of test cases that maximizes a number of coverage points in the universe of coverage points being tested at least once.
2 . The method of claim 1 , wherein the weights associated with each test case of the plurality of test cases comprise weights associated with a number of times the test case was executed and how the test case contributes to ensuring that at least one coverage point is hit during execution of the test case.
3 . The method of claim 1 , wherein each coverage point is associated with a probability of hitting the coverage point for each test case of the plurality of test cases, the probability being calculated based on a binomial distribution over historical test case execution data for each coverage point in the universe of coverage points.
4 . The method of claim 1 , wherein the machine learning model is implemented by a single-layer neural network.
5 . The method of claim 4 , wherein the single-layer neural network reduces a plurality of input nodes associated with vectors representing the universe of coverage points to a single output node for the single-layer neural network.
6 . The method of claim 1 , wherein the machine learning model comprises a model having been trained based on backpropagation of a hinge loss value associated with the plurality of test cases and a distance value weighted based on a cost factor associated with each test case of the plurality of test cases.
7 . The method of claim 6 , wherein the cost factor is set to 1 in order to minimize a number of test cases included in the selected subset of test cases.
8 . The method of claim 6 , wherein the cost factor associated with each test case of the plurality of test cases is associated with a computational expense incurred in executing each test case of the plurality of test cases.
9 . The method of claim 1 , wherein the minimal number of test cases that maximizes the number of coverage points being tested at least once is selected based on a computational cost threshold.
10 . A system, comprising:
a memory having executable instructions stored thereon; and
a processor configured to execute the executable instructions in order to cause the system to:
generate a coverage matrix associated with a plurality of test cases for an integrated circuit and coverage points in a universe of coverage points associated with each test case of the plurality of test cases;
select a subset of the plurality of test cases for execution on the integrated circuit based on weights associated with each test case of the plurality of test cases and a threshold weight value, wherein the weights associated with each test case of the plurality of test cases comprise weights in a machine learning model trained based on the coverage matrix; and
test the integrated circuit based on the selected subset of test cases;
wherein in order to select the subset of the plurality of test cases for execution, the processor is configured to cause the system to select a minimal number of test cases that maximizes a number of coverage points in the universe of coverage points being tested at least once.
11 . The system of claim 10 , wherein the weights associated with each test case of the plurality of test cases comprise weights associated with a number of times the test case was executed and how the test case contributes to ensuring that at least one coverage point is hit during execution of the test case.
12 . The system of claim 10 , wherein each coverage point is associated with a probability of hitting the coverage point for each test case of the plurality of test cases, the probability being calculated based on a binomial distribution over historical test case execution data for each coverage point in the universe of coverage points.
13 . The system of claim 10 , wherein the machine learning model is implemented by a single-layer neural network.
14 . The system of claim 13 , wherein the single-layer neural network reduces an input data set of nodes associated with vectors representing the universe of coverage points to a single output node for the single-layer neural network.
15 . The system of claim 10 , wherein the machine learning model comprises a model having been trained based on backpropagation of a hinge loss value associated with the plurality of test cases and a distance value weighted based on a cost factor associated with each test case of the plurality of test cases.
16 . The system of claim 15 , wherein the cost factor is set to 1 in order to minimize a number of test cases included in the selected subset of test cases.
17 . The system of claim 15 , wherein the cost factor associated with each test case of the plurality of test cases is associated with a computational expense incurred in executing each test case of the plurality of test cases.
18 . The system of claim 11 , wherein the minimal number of test cases that maximizes the number of coverage points being tested at least once is selected based on a computational cost threshold.
19 . A system, comprising:
means for generating a coverage matrix associated with a plurality of test cases for an integrated circuit and coverage points in a universe of coverage points associated with each test case of the plurality of test cases;
means for selecting a subset of the plurality of test cases for execution on the integrated circuit based on weights associated with each test case of the plurality of test cases and a threshold weight value, wherein the weights associated with each test case of the plurality of test cases comprise weights in a machine learning model trained based on the coverage matrix; and
means for testing the integrated circuit based on the selected subset of test cases;
wherein the means for selecting the subset of the plurality of test cases for execution comprises selecting a minimal number of test cases that maximizes a number of coverage points in the universe of coverage points being tested at least once.
20 . The system of claim 19 , wherein the weights associated with each test case of the plurality of test cases comprise weights associated with a number of times the test case was executed and how the test case contributes to ensuring that at least one coverage point is hit during execution of the test case.
21 . The system of claim 19 , wherein each coverage point is associated with a probability of hitting the coverage point for each test case of the plurality of test cases, the probability being calculated based on a binomial distribution over historical test case execution data for each coverage point in the universe of coverage points.
22 . The system of claim 19 , wherein the machine learning model is implemented by a single-layer neural network.
23 . The system of claim 22 , wherein the single-layer neural network reduces an input data set of nodes associated with vectors representing the universe of coverage points to a single output node for the single-layer neural network.
24 . The system of claim 19 , wherein the machine learning model comprises a model having been trained based on backpropagation of a hinge loss value associated with the plurality of test cases and a distance value weighted based on a cost factor associated with each test case of the plurality of test cases.
25 . The system of claim 24 , wherein the cost factor is set to 1 in order to minimize a number of test cases included in the selected subset of test cases.
26 . The system of claim 24 , wherein the cost factor associated with each test case of the plurality of test cases is associated with a computational expense incurred in executing each test case of the plurality of test cases.
27 . A computer-readable medium having instructions stored thereon which, when executed by at least one processor, cause the at least one processor to perform an operation comprising:
generating a coverage matrix associated with a plurality of test cases for an integrated circuit and coverage points in a universe of coverage points associated with each test case of the plurality of test cases;
selecting a subset of the plurality of test cases for execution on the integrated circuit based on weights associated with each test case of the plurality of test cases and a threshold weight value, wherein the weights associated with each test case of the plurality of test cases comprise weights in a machine learning model trained based on the coverage matrix; and
testing the integrated circuit based on the selected subset of test cases;
wherein selecting the subset of the plurality of test cases for execution comprises selecting a minimal number of test cases that maximizes a number of coverage points in the universe of coverage points being tested at least once.