Apparatus and method for simulation automation in regression test
A method of simulating an integrated circuit includes providing at least one test case to a simulation tool, obtaining at least one first simulation result and at least one first simulation log from the simulation tool, classifying, with a first machine learning model, the at least one test case into one fail class of a plurality of fail classes, generating at least one renewed test case by applying, with a controller, a solution to the at least one test case, and providing the at least one renewed test case to the simulation tool.
1 . A method of simulating an integrated circuit, the method comprising:
providing at least one test case to a simulation tool;
obtaining at least one first simulation result and at least one first simulation log from the simulation tool;
outputting, with a first machine learning model, a probability that the at least one test case is to be classified into at least one fail class;
classifying, with the first machine learning model, the at least one test case into one fail class of a plurality of fail classes based on the probability that the at least one test case is to be classified into at least one fail class being greater than a predetermined threshold probability value;
generating at least one renewed test case by applying, with a controller, a solution to the at least one test case; and
providing the at least one renewed test case to the simulation tool.
2 . The method of claim 1 , further comprising:
providing a first test case set to a second machine learning model; and
predicting, with the second machine learning model, at least one test case of the first test case set where a simulation result is expected to fail.
3 . The method of claim 2 , further comprising:
generating, with the simulation tool, a second simulation result based on a second test case set; and
training the second machine learning model based on the second test case set and the second simulation result.
4 . The method of claim 2 , wherein the second machine learning model comprises a third sub-model configured to:
receive the first test case set;
vectorize the first test case set; and
generate second data based on the vectorized first test case set.
5 . The method of claim 4 , wherein the first test case set comprises at least one attribute, and
wherein the at least one attribute comprises a target, a test class, and a function target.
6 . The method of claim 4 , wherein the second machine learning model comprises a fourth sub-model configured based on logistic regression and configured to determine whether a simulation result of the first test case set fails or passes based on the second data.
7 . The method of claim 1 , wherein the first machine learning model comprises a first sub-model configured to:
receive the at least one first simulation log;
tokenize the at least one first simulation log, and
generate first data based on the tokenized at least one first simulation log.
8 . The method of claim 7 , wherein the first sub-model comprises:
a model based on a transformer, and
at least one of a byte pair encoding (BPE) algorithm and a wordpiece algorithm.
9 . The method of claim 7 , wherein the first machine learning model further comprises a second sub-model comprising a fully connected layer and configured to receive the first data from the first sub-model and output the probability of the at least one test case to be classified into at least one fail class based on the first data.
10 . The method of claim 9 , wherein the at least one fail class comprises a data mismatch occurring in source data.
11 . The method of claim 1 , wherein the at least one fail class comprises a data mismatch, and the solution comprises at least one of version management and a system-on design revision based on the data mismatch.
12 . A system comprising:
at least one memory storing instructions; and
at least one processor configured to execute the instructions to:
provide at least one test case to a simulation tool;
obtain at least one first simulation result and at least one first simulation log from the simulation tool;
output, with a first machine learning model, a probability that the at least one test case is to be classified into at least one fail class;
classify, with the first machine learning model, the at least one test case into one fail class of a plurality of fail classes based on the probability that the at least one test case is to be classified into at least one fail class being greater than a predetermined threshold probability value;
generate a renewed test case by applying, with a controller, a solution to the at least one test case; and
provide the renewed test case to the simulation tool.
13 . The system of claim 12 , wherein the at least one processor is further configured to execute the instructions to:
provide a first test case set to a second machine learning model; and
predict, with the second machine learning model, at least one test case of the first test case set where a simulation result is expected to fail.
14 . The system of claim 13 , wherein the at least one processor is further configured to execute the instructions to:
generate, with the simulation tool, a second simulation result based on a second test case set; and
train the second machine learning model based on the second test case set and the second simulation result.
15 . The system of claim 13 , wherein the second machine learning model comprises:
a third sub-model configured to:
receive the first test case set;
vectorize the first test case set; and
generate second data based on the vectorized first test case set; and
a fourth sub-model configured based on logistic regression and configured to determine whether a simulation result of the first test case set fails or passes based on the second data.
16 . The system of claim 12 , wherein the first machine learning model comprises:
a first sub-model configured to:
receive the at least one first simulation log;
tokenize the at least one first simulation log; and
generate first data based on the tokenized at least one first simulation log; and
a second sub-model comprising a fully connected layer, the second sub-model configured to receive the first data from the first sub-model and output the probability of the at least one test case to be classified into at least one fail class based on the first data.
17 . A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to:
provide at least one test case to a simulation tool;
obtain at least one first simulation result and at least one first simulation log from the simulation tool;
output, with a first machine learning model, a probability that the at least one test case is to be classified into at least one fail class;
classify, with the first machine learning model, the at least one test case into one fail class of a plurality of fail classes based on the probability that the at least one test case is to be classified into at least one fail class being greater than a predetermined threshold probability value;
generate a renewed test case by applying, with a controller, a solution to the at least one test case; and
provide the renewed test case to the simulation tool.
18 . The non-transitory computer-readable storage medium of claim 17 , wherein the instructions, when executed by the at least one processor, further cause the at least one processor to:
provide a first test case set to a second machine learning model, and
predict, with the second machine learning model, at least one test case of the first test case set where a simulation result is expected to fail.
19 . The non-transitory computer-readable storage medium of claim 18 , wherein the second machine learning model comprises:
a third sub-model configured to:
receive the first test case set;
vectorize the first test case set; and
generate second data based on the vectorized first test case set; and
a fourth sub-model configured based on logistic regression and configured to determine whether a simulation result of the first test case set fails or passes based on the second data.
20 . The non-transitory computer-readable storage medium of claim 17 , wherein the first machine learning model comprises:
a first sub-model configured to:
receive the at least one first simulation log;
tokenize the at least one first simulation log; and
generate first data based on the tokenized at least one first simulation log; and
a second sub-model comprising a fully connected layer, the second sub-model being configured to receive the first data from the first sub-model and output the probability of the at least one test case to be classified into at least one fail class based on the first data.