Software and hardware emulation system
A method to emulate a system represented by one or more of hardware portions and software portions is described. The method comprises determining whether a subset of the one or more hardware portions and software portions have been tested, and identifying whether the system has performed to a specification based on the testing. The method further comprising, when the system has not performed to the specification, determining one or more of the hardware and software portions to update for retesting.
1. A method comprising:
demarcating, by at least one computer processor, a plurality of code portions as a function of time, with respect to a predetermined time period;
evaluating, by the at least one computer processor, a result of a dynamic test, wherein the dynamic test comprises an automated runtime observation;
filtering, by the at least one computer processor, the plurality of code portions, by automatically identifying a first code portion to omit from testing via a filter generated in accordance with a test policy specification, based at least in part on the automated runtime observation, and based at least in part on a determination of whether the runtime observation occurs in a range of time with respect to the predetermined time period;
testing, by the at least one computer processor, a second code portion of the plurality of code portions, using the dynamic test;
capturing, by the at least one computer processor, a test result indicating at least the second code portion, tested using the dynamic test;
identifying, by the at least one computer processor, a subset of the plurality of code portions to check, by the at least one computer processor, in an abstract syntax tree (AST) representing a conditional branch code path, wherein the AST is a result of a static analysis of the subset of the plurality of code portions;
determining, by the at least one computer processor, adequacy of the test result based at least in part on the runtime observation, based at least in part on checking the identified subset of the plurality of code portions in the AST, and based at least in part on analyzing output resulting from conditional execution of a code statement corresponding to at least the first code portion or the second code portion; and
outputting, by the at least one computer processor, an indication of whether the dynamic test meets a requirement, based on the determined adequacy of the test result.
2. The method of claim 1 , wherein the subset of the plurality of code portions is determined by identifying a plurality of lines of code.
3. The method of claim 1 , wherein the subset of the plurality of code portions is determined by identifying specific values of code variables.
4. The method of claim 1 , wherein the subset of the plurality of code portions is determined by identifying a specific sequence of code statements.
5. The method of claim 1 , wherein the subset of the plurality of code portions is determined by one or more functions within the plurality of code portions.
6. The method of claim 1 ,
wherein the test result comprises a conditional execution sequence.
7. The method of claim 1 , further comprising:
using static analysis of the plurality of code portions to identify the subset of the plurality of code portions to include in the dynamic test.
8. A system comprising memory and at least one processor configured to:
demarcate a plurality of code portions as a function of time, with respect to a predetermined time period;
evaluate a result of a dynamic test, wherein the dynamic test comprises an automated runtime observation;
filter the plurality of code portions, by automatically identifying a first code portion to omit from testing via a filter generated in accordance with a test policy specification, based at least in part on the automated runtime observation, and based at least in part on a determination of whether the runtime observation occurs in a range of time with respect to the predetermined time period;
test a second code portion of the plurality of code portions, using the dynamic test;
capture a test result indicating at least the second code portion, tested using the dynamic test;
identify a subset of the plurality of code portions to check in an abstract syntax tree (AST) representing a conditional branch code path, wherein the AST is a result of a static analysis of the subset of the plurality of code portions;
determine adequacy of the test result based at least in part on the runtime observation, based at least in part on checking the identified subset of the plurality of code portions in the AST, and based at least in part on analyzing output resulting from conditional execution of a code statement corresponding to at least the first code portion or the second code portion; and
output an indication of whether the dynamic test meets a requirement, based on the determined adequacy of the test result.
9. The system of claim 8 , wherein the subset of the plurality of code portions is determined by identifying specific values of code variables.
10. The system of claim 8 , wherein the subset of the plurality of code portions is determined by identifying a specific sequence of code statements.
11. The system of claim 8 , wherein the subset of the plurality of code portions is determined by one or more functions within the plurality of code portions.
12. The system of claim 8 , wherein the test result comprises a conditional execution sequence.
13. The system of claim 8 , the at least one processor further configured to:
use static analysis of the plurality of code portions to identify the subset of the plurality of code portions to include in the dynamic test.
14. A non-transitory machine-readable storage device medium comprising instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:
demarcating a plurality of code portions as a function of time, with respect to a predetermined time period;
evaluating a result of a dynamic test, wherein the dynamic test comprises an automated runtime observation;
filtering the plurality of code portions, by automatically identifying a first code portion to omit from testing via a filter generated in accordance with a test policy specification, based at least in part on the automated runtime observation, and based at least in part on a determination of whether the runtime observation occurs in a range of time with respect to the predetermined time period;
testing a second code portion of the plurality of code portions, using the dynamic test;
capturing a test result indicating at least the second code portion, tested using the dynamic test;
identifying a subset of the plurality of code portions to check in an abstract syntax tree (AST) representing a conditional branch code path, wherein the AST is a result of a static analysis of the subset of the plurality of code portions;
determining adequacy of the test result based at least in part on the runtime observation, based at least in part on checking the identified subset of the plurality of code portions in the AST, and based at least in part on analyzing output resulting from conditional execution of a code statement corresponding to at least the first code portion or the second code portion; and
outputting an indication of whether the dynamic test meets a requirement, based on the determined adequacy of the test result.
15. The non-transitory machine-readable storage device medium of claim 14 , wherein the subset of the plurality of code portions is determined by identifying a plurality of lines of code.
16. The non-transitory machine-readable storage device medium of claim 14 , wherein the subset of the plurality of code portions is determined by identifying specific values of code variables.
17. The non-transitory machine-readable storage device medium of claim 14 , wherein the subset of the plurality of code portions is determined by identifying a specific sequence of code statements.
18. The non-transitory machine-readable storage device medium of claim 14 , wherein the subset of the plurality of code portions is determined by one or more functions within the plurality of code portions.
19. The non-transitory machine-readable storage device medium of claim 14 , wherein the test result comprises a conditional execution sequence.
20. The non-transitory machine-readable storage device medium of claim 14 , the operations further comprising:
using static analysis of the plurality of code portions to identify the subset of the plurality of code portions to include in the dynamic test.