Attribute relevance tagging in malware recognition
In general, in one aspect, a method for machine learning recognition of portable executable files as malware includes providing training data comprising features of portable executable files and a descriptive information for the portable executable files, the descriptive information comprising a family or type of malware. The method may include training a model using the training data to detect malware. The method may include using the trained model to recognize malware by providing features of a portable executable file as input and providing a threat score and descriptive information as output.
1. A method for machine learning recognition of portable executable files as malware, the method comprising:
providing training data comprising features of portable executable files and an attribute indicator for the portable executable files, the attribute indicator comprising a family or type of malware;
training a model using the training data to detect malware;
using the trained model to recognize malware by providing features of a portable executable file as input and providing a threat score and an attribute indicator as output; and
taking a remedial action based on the output,
wherein the family or type of malware is determined for the training data by:
parsing detection names to extract relevant tokens for each detection name based on sequences of characters within that detection name;
filtering the tokens based on prevalence by removing tokens appearing in a fraction of samples of the training data below a predetermined threshold; and
mapping the filtered tokens to tags using one or more association rules.
2. The method of claim 1 , wherein the training further comprises training a neural network model using portable executable files and tags in a joint embedding model thereby generating a latent space.
3. The method of claim 1 , wherein the trained model is a neural network.
4. The method of claim 1 , wherein the remedial action includes at least one of quarantining the file, notifying a user or administrator that the file is malicious, displaying an indication that the file is malicious, displaying an indication of the type of file, or removing the file.
5. The method of claim 1 , wherein the attribute indicator includes at least one type of attribute indicator selected from the list of: adware, crypto-miner, downloader, dropper, fileinfector, flooder, installer, packed, ransomware, spyware, and worm.
6. The method of claim 1 , wherein the features are determined using a feature extractor.
7. The method of claim 1 , wherein the malware type is determined by distilling a detection name to provide labels.
8. The method of claim 7 , wherein the distilling comprises determining attributes of a malware file.
9. The method of claim 1 , wherein the training data is generated using static detections and behavior analysis.
10. The method of claim 1 , wherein the tags include at least one tag for one or more of adware, a downloader, ransomware, a packed file, and a file-infector.
11. A non-transitory processor-readable medium storing code representing instructions to be executed by a processor, the code comprising executable instructions to cause the processor to:
provide training data comprising features of artifacts and an attribute indicator for the artifacts, the attribute indicator comprising a type of artifact;
train a machine learning model using the training data to detect malware; and
use the trained machine learning model to recognize malware by providing features of an artifact as input and providing both a threat score and an attribute indicator of the type of artifact as output,
wherein the type of artifact is determined for the training data by:
parsing detection names to extract relevant tokens for each detection name based on sequences of characters within that detection name;
filtering the tokens based on prevalence by removing tokens appearing in a fraction of samples of the training data below a predetermined threshold; and
mapping the filtered tokens to tags using one or more association rules.
12. The medium of claim 11 , wherein the artifact is at least one of a portable executable file, a script, a Hypertext Markup Language (HTML) file, a JavaScript file, or a Hypertext Preprocessor (PHP) file.
13. The medium of claim 11 , wherein the machine learning model is a neural network.
14. The medium of claim 11 , further comprising taking a remedial action based on the output.
15. The medium of claim 14 , wherein the remedial action includes at least one of quarantining the artifact, notifying a user or administrator that the artifact is malicious, displaying an indication that the artifact is malicious, displaying an indication of the type of artifact, or removing the artifact.
16. The medium of claim 11 , wherein the features are determined using a feature extractor and the artifact type is determined by distilling a detection name to provide labels.
17. The medium of claim 16 , wherein the distilling comprises determining attributes of a detected artifact.
18. The medium of claim 11 , wherein the training data is generated using static detections and behavior analysis.
19. The medium of claim 11 , wherein the tags include one or more tags for types of malware.
20. The medium of claim 11 , wherein the tags include at least one tag for one or more of adware, a downloader, ransomware, a packed file, and a file-infector.