Malware detection using machine learning
Synthetic training sets for machine learning are created by identifying and modifying functional features of code in an existing malware training set. By filtering the resulting synthetic code to measure malware impact and novelty, training sets can be created that predict novel malware and to seek to preemptively exhaust the space of new malware. These synthesized training sets can be used in turn to improve training of machine learning models. Furthermore, by repeating the process of new code generation, filtering and training, an iterative machine learning process may be created that continuously narrows the window of vulnerabilities to new malicious actions.
1. A computer program product comprising computer executable code embodied in a non-transitory computer readable medium that, when executing on one or more computing devices, performs the steps of:
providing a first training set including a plurality of malware samples;
configuring a first antimalware system to detect the malware samples;
characterizing one or more functional blocks of the malware samples;
generating a first number of synthetic malware samples including modifications of the one or more functional blocks of the malware samples;
validating the first number of synthetic malware samples to provide a validated sample set containing one or more of the first number of synthetic malware samples that execute and perform an unwanted task in a target computing context;
filtering the validated sample set to provide a filtered sample set containing one or more of the first number of synthetic malware samples in the validated sample set that are not detected by the first antimalware system;
creating a second antimalware system by training a machine learning malware detection engine to detect malicious code including the one or more of the first number of synthetic malware samples in the validated sample set;
generating a second number of synthetic malware samples based on functional blocks of the first number of synthetic malware samples in the validated sample set;
validating and filtering the second number of synthetic malware samples to provide a second training set; and
creating a third antimalware system by training the machine learning malware detection engine to detect synthetic malware samples in the second training set.
2. A method comprising:
providing a first training set including a plurality of malware samples;
configuring a first antimalware system to detect the malware samples;
generating a first number of synthetic malware samples including modifications of the malware samples;
validating the first number of synthetic malware samples to provide a validated sample set containing one or more of the first number of synthetic malware samples that execute and perform an unwanted task in a target computing context;
filtering the validated sample set to provide a filtered sample set containing one or more of the first number of synthetic malware samples in the validated sample set that are not detected by the first antimalware system;
creating a second antimalware system by training a machine learning malware detection engine to detect malicious code including the one or more of the first number of synthetic malware samples in the validated sample set;
generating a second number of synthetic malware samples based on functional blocks of the first number of synthetic malware samples in the validated sample set;
validating and filtering the second number of synthetic malware samples to provide a second training set; and
creating a third antimalware system by training the machine learning malware detection engine to detect synthetic malware samples in the second training set.
3. The method of claim 2 wherein generating the first number of synthetic malware samples includes generating code with a generative engine of a generative adversarial network.
4. The method of claim 2 wherein generating the first number of synthetic malware samples includes testing generated samples with a discriminative engine of a generative adversarial network.
5. The method of claim 2 further comprising characterizing one or more functional blocks of the malware samples.
6. The method of claim 2 wherein generating the first number of synthetic malware samples includes applying a fuzzer to one or more of the plurality of malware samples in the first training set.
7. The method of claim 2 wherein generating the first number of synthetic malware samples includes applying a randomizer to one or more of the plurality of malware samples in the first training set.
8. The method of claim 2 wherein validating the first number of synthetic malware samples includes executing each of the first number of synthetic malware samples in a sandbox.
9. The method of claim 2 wherein filtering includes removing any of the first number of synthetic malware samples substantially identical to one of the plurality of malware samples.
10. The method of claim 2 wherein filtering includes removing any one of the first number of synthetic malware samples substantially identical to another one of the first number of synthetic malware samples.
11. The method of claim 2 further comprising iteratively repeating the generating, validating, filtering and creating a new antimalware system until a predetermined threshold is reached.
12. The method of claim 2 further comprising deploying the second antimalware system on an endpoint.
13. The method of claim 2 further comprising deploying the second antimalware system on a gateway to an enterprise network.
14. The method of claim 2 further comprising deploying the second antimalware system at a threat management facility for an enterprise network.
15. The method of claim 2 further comprising using the second antimalware system to detect malware in an enterprise network.
16. The method of claim 2 further comprising detecting malware on an endpoint in an enterprise network with the second antimalware system and initiating a remediation of the endpoint.
17. The method of claim 5 wherein generating the first number of synthetic malware samples includes generating code based on known variants of the one or more functional blocks.
18. The method of claim 5 wherein generating the first number of synthetic malware samples includes generating code based on known substitutes for the one or more functional blocks.
19. The method of claim 11 wherein the predetermined threshold includes a receiver operating characteristic for the new antimalware system.
20. A system comprising:
a threat management facility including a first memory storing computer-executable code for deploying a machine learning engine configured to train a detection model for an antimalware system by training a detection model to detect malicious code in one or more of a number of malware samples in a training set, generating synthetic malware samples including modifications of one or more functional blocks of the number of malware samples in the training set, filtering the synthetic malware samples to update to the number of malware samples in the training set with synthetic malware samples that are undetectable by the detection model and operable, and repeating one or more of training, generating, and filtering until a receiver operating characteristic for the detection model achieves a predetermined threshold for true positive detection; and
an endpoint coupled to an enterprise network, the endpoint including a second memory storing computer-executable code for deploying the detection model to detect malicious code received through the enterprise network.