Training classifiers for program analysis
View Patent ↗Classifier training modules and systems are shown that include a memory configured to store a known vulnerability distribution and an initial feature set; and a processor configured to run an initial security analysis on a training codebase to generate a set of vulnerabilities associated with the training codebase, to analyze the program with the feature set to limit a number of detected vulnerabilities to generate a limited set of vulnerabilities associated with the feature set, to compare the limited set of vulnerabilities to the known vulnerability distribution to generate an accuracy score, and to iteratively refine the analysis by updating the feature set to find a feature set having a highest accuracy score.
1. A classifier training module, comprising:
a memory configured to store a known vulnerability distribution and an initial feature set; and
a processor configured to run an initial security analysis on a training codebase to generate a set of vulnerabilities associated with the training codebase, to analyze a program with the initial feature set to limit a number of detected vulnerabilities to generate a limited set of vulnerabilities associated with the initial feature set, to compare the limited set of vulnerabilities to the known vulnerability distribution to generate an accuracy score, and to iteratively refine the analysis by updating the feature set to find a feature set having a highest accuracy score.
2. The classifier training module of claim 1 , wherein the feature set includes vulnerability path length.
3. The classifier training module of claim 1 , wherein the feature set includes one or more blocked code locations.
4. The classifier training module of claim 1 , wherein the known vulnerability distribution comprises a statistical representation of vulnerability prevalence.
5. The classifier training module of claim 1 , wherein the processor is further configured to compare numbers of vulnerabilities per line of code.
6. The classifier training module of claim 1 , wherein the processor is further configured to generate a classifier based on the feature set having a highest accuracy score.
7. The classifier training module of claim 1 , wherein the training codebase comprises a first set of applications and wherein the known vulnerability distribution is based on a manual analysis of a second set of applications.
8. The classifier training module of claim 7 , wherein the training codebase further comprises a user's local codebase.
9. The classifier training module of claim 1 , wherein each iteration uses a different feature set based on a Tabu search strategy.
10. A classifier training module, comprising:
a memory configured to store a known statistical representation of vulnerability prevalence and an initial feature set comprising vulnerability path length; and
a processor configured to run an initial security analysis on a training codebase to generate a set of vulnerabilities associated with the training codebase, to analyze the training codebase with a feature set that limits a number of detected vulnerabilities to generate a limited set of vulnerabilities associated with the feature set, to compare a number of vulnerabilities per line of code found with the limited set of vulnerabilities to the known statistical representation of vulnerability prevalence to generate an accuracy score using a processor, to iteratively refine the analysis by updating the feature set to find a feature set having a highest accuracy score, and to generate a classifier based on the feature set having a highest accuracy score.
11. The classifier training module of claim 10 , wherein the feature set includes one or more blocked code locations.
12. The classifier training module of claim 10 , wherein the training codebase comprises a first set of applications and wherein the known vulnerability distribution is based on a manual analysis of a second set of applications.
13. The classifier training module of claim 10 , wherein the training codebase further comprises a user's local codebase.
14. A static security analysis system, comprising:
a classifier training module, comprising:
a memory configured to store a known vulnerability distribution and an initial feature set; and
a processor configured to run an initial security analysis on a training codebase to generate a set of vulnerabilities associated with the training codebase, to analyze a program with the initial feature set to limit a number of detected vulnerabilities to generate a limited set of vulnerabilities associated with the initial feature set, to compare the limited set of vulnerabilities to the known vulnerability distribution to generate an accuracy score, to iteratively refine the analysis by updating the feature set to find a feature set having a highest accuracy score, and to generate a classifier based on the feature set having the highest accuracy score; and
a static analysis module configured to scan code using the classifier to locate potential vulnerabilities.
15. The static security analysis system of claim 14 , wherein the feature set includes vulnerability path length.
16. The static security analysis system of claim 14 , wherein the feature set includes one or more blocked code locations.
17. The static security analysis system of claim 14 , wherein the known vulnerability distribution comprises a statistical representation of vulnerability prevalence.
18. The static security analysis system of claim 14 , wherein the processor is further configured to compare numbers of vulnerabilities per line of code.
19. The static security analysis system of claim 14 , wherein the training codebase comprises a first set of applications and wherein the known vulnerability distribution is based on a manual analysis of a second set of applications.
20. The static security analysis system of claim 19 , wherein the training codebase further comprises a user's local codebase.
21. The static security analysis system of claim 14 , wherein each iteration uses a different feature set based on a Tabu search strategy.
22. A static security analysis system, comprising:
a classifier training module, comprising:
a memory configured to store a known statistical representation of vulnerability prevalence and an initial feature set comprising vulnerability path length; and
a processor configured to run an initial security analysis on a training codebase to generate a set of vulnerabilities associated with the training codebase, to analyze the training codebase with a feature set that limits a number of detected vulnerabilities to generate a limited set of vulnerabilities associated with the feature set, to compare a number of vulnerabilities per line of code found with the limited set of vulnerabilities to the known statistical representation of vulnerability prevalence to generate an accuracy score using a processor, to iteratively refine the analysis by updating the feature set to find a feature set having a highest accuracy score, and to generate a classifier based on the feature set having a highest accuracy score; and
a static analysis module configured to scan code using the classifier to locate potential vulnerabilities.
23. A non-transitory computer readable storage medium comprising a computer readable program for training a classifier, wherein the computer readable program when executed on a computer causes the computer to perform the steps of:
running an initial security analysis on a training codebase to generate a set of vulnerabilities associated with the training codebase;
analyzing a program with a feature set that limits a number of detected vulnerabilities to generate a limited set of vulnerabilities associated with the feature set;
comparing the limited set of vulnerabilities to a known vulnerability distribution to generate an accuracy score using a processor; and
iterating said steps of analyzing and comparing using different feature sets to find a feature set having a highest accuracy score.