Accelerating static program analysis with artifact reuse
A system performs static program analysis with artifact reuse. The system identifies artifacts associated with the software program being analyzed. The system processes the identified artifacts for performing static program analysis and transmits either the artifacts or identifiers for the artifacts to a second processing device for performing program analysis. The second processing device receives the artifacts and uses the received identifiers to retrieve the artifacts from a networked storage system. The second device also retrieves stored summaries of previous program analysis from the networked storage system. The program analysis uses the retrieved artifacts to generate work units for static program analysis. The analysis is performed only for those work units that are determined to remain unchanged from previous static program analysis cycles.
1. A system comprising:
a memory storing instructions; and
a processor device, coupled with the memory and to execute the instructions, the instructions when executed cause the processor device to:
gather artifacts for a software program, wherein the artifacts are used in a static interprocedural analysis of the software program;
determine whether gathered artifacts are eligible for artifact reuse;
identify at least one of the artifacts determined to be eligible for artifact reuse as having a corresponding record stored in a network-accessible artifact repository; and
generate an emit database for the software program, comprising: storing an artifact identifier for the identified artifact in the emit database.
2. The system of claim 1 wherein the artifacts represented in the emit database comprise software code and libraries, and the emit database contains contents of artifacts, derivatives of artifacts, and artifact identifiers.
3. The system of claim 2 wherein the derivatives of artifacts comprise transformations of artifacts into a form suitable for the static analysis.
4. The system of claim 2 wherein at least one artifact is source code, and the transformation of the source code artifact comprises performing syntactical and semantical analysis of the source code artifact.
5. The system of claim 1 wherein the static interprocedural analysis comprises detecting defects and/or vulnerabilities in the software program.
6. A method comprising:
gathering artifacts for a software program, wherein the artifacts are used in a static analysis of the software program;
determining whether gathered artifacts are eligible for artifact reuse;
identifying at least one of the artifacts determined to be eligible for artifact reuse as having a corresponding record stored in a network-accessible artifact repository; and
generating, by a capture processing device, an emit database for the software program, comprising:
storing an artifact identifier for the identified artifact in the emit database.
7. The method of claim 6 wherein the emit database contains contents of artifacts, decompilation artifacts, and artifact identifiers; and the decompilation artifacts result from applying an artifact decompilation process to compiled artifacts.
8. The method of claim 6 wherein identifying at least one of the artifacts determined to be eligible for artifact reuse as having a corresponding record comprises:
calculating fingerprints for the artifacts determined to be eligible for artifact reuse; and
comparing the fingerprints to fingerprints stored in the records of the artifact repository.
9. The method of claim 6 wherein the corresponding record in the artifact repository contains the content of at least one of the artifacts determined to be eligible for artifact reuse, and the emit database does not contain that content.
10. The method of claim 6 wherein the corresponding record in the artifact repository contains a decompilation artifact of the at least one artifact, and the emit database does not contain that decompilation artifact.
11. The method of claim 6 further comprising:
for at least one of the artifacts determined to be eligible for artifact reuse without a corresponding record stored in the artifact repository, creating a record for that at least one artifact in the artifact repository.
12. The method of claim 11 wherein the created record contains content of the at least one artifact, and at least one derivative of the at least one artifact.
13. The method of claim 6 further comprising:
based on the artifacts represented in the emit database, partitioning the static analysis into a plurality of work units;
identifying at least one of the work units as having a corresponding record stored in a network-accessible summaries repository, the record including a summary of results for analysis specified by the work unit;
retrieving the summary from the summaries repository; and
storing the retrieved summary in a results database for the static analysis for the software program.
14. The method of claim 13 further comprising:
for artifact identifiers in the emit database, retrieving the corresponding artifacts from the artifact repository, wherein partitioning the static analysis into a plurality of work units is also based on the retrieved artifacts.
15. A system comprising:
a network-accessible artifact repository that stores artifacts;
a capture processing device configured to capture artifacts for a software program and to generate an emit database for the software program from the captured artifacts; wherein the captured artifacts are used in a static analysis of the software program; and
an analysis processing device configured to receive the emit database from the capture processing device and to perform the static analysis of the software program using the artifacts represented in the emit database, including retrieving artifacts stored in the artifact repository, and to generate a results database of the static analysis for the software programs wherein the capture processing device is configured to generate individual emit databases for each of a plurality of software programs, and the analysis processing device is configured to generate individual results databases for each of the plurality of software programs.
16. The system of claim 15 wherein sections of the artifact repository corresponding to libraries are accessible to the capture processing device when generating the emit databases for any of the plurality of software programs.
17. The system of claim 15 wherein records stored in the artifact repository include software program identifiers, and the software program identifiers are used to restrict access to the stored records.
18. The system of claim 17 wherein records stored in the artifact repository further include software program version identifiers.
19. The system of claim 15 wherein the capture processing device performs the static analysis incrementally for a new version of the software program, and the artifact repository is updated according to the incremental performance of the static analysis.
20. The system of claim 19 wherein the incremental static analysis for new versions of the software program comprises:
retrieving the emit database for a prior version of the software program;
updating the emit database for artifacts that have changed from the prior version to the new version; and
performing the static analysis using the artifacts represented in the updated emit database.