Machine learning-based malicious attachment detector

1. A computer-implemented method comprising:

receiving, from a pre-processor, a first output file;

wherein the first output file is created by the pre-processor in response to input of a first electronic file to the pre-processor, wherein the first output file indicates that the first electronic file is a Portable Document Format (PDF) file;

wherein the first electronic file is an attachment to a first message that is in-transit to a first recipient computer on a network;

wherein the first output file contains features that are created by the pre-processor analyzing one or more sub-features of the first electronic file and adjusting the features by one or more of: reducing the features using mutual information criteria and transforming the features to a floating point vector;

receiving, from a machine learning-based classifier, malware classification data that indicates whether the first electronic file does or does not contain malware;

wherein the malware classification data is output by the machine learning-based classifier in response to the machine learning-based classifier determining that the features are or are not indicators of obfuscation, and wherein the machine learning-based classifier is created using a neural network and the malware classification data is produced by inputting a single image of the first electronic file to the machine learning-based classifier;

wherein data used to create the machine learning-based classifier includes output files previously created by the pre-processor; and

in response to the malware classification data matching a criterion, causing the network to modify, delay, or block transmission of the first electronic file to the first recipient computer.

2. The method of claim 1 , wherein the pre-processor performs any one or more of the following to create the first output file: parsing the first electronic file, pre-execution static analysis of the first electronic file, extracting script components from the first electronic file.

3. The method of claim 1 , wherein the features comprises any one or more of the following: an occurrence count, an occurrence frequency, metadata, a string of interest, a warning, a classification, a rule name.

4. The method of claim 1 , wherein the one or more sub-features of the first electronic file comprises any one or more of the following: a version number, an image, a video, a color, a string of interest, an embedded object, an object identifier, a trailer object, a stream, an end of file (EOF) indicator, a variable, an xref tag, a startxref tag, a Uniform Resource Locators (URL), a metadata item.

5. The method of claim 1 , wherein a model used by the machine learning-based classifier is created using any one or more of the following: a tree-based algorithm, a random forest algorithm, a deep learning algorithm, a neural network, a deep convolutional neural network.

6. The method of claim 1 , further comprising:

creating, by the pre-processor, a second output file based on input of a second electronic file, wherein the second output file indicates that the second electronic file contains Visual Basic for Application (VBA) code or programming code, wherein the second electronic file is a second attachment to a second message that is in-transit to a second recipient computer on the network, and wherein the second output file includes features that are created by the pre-processor analyzing one or more sub-features of the second electronic file, wherein the features comprises any one or more of the following: a word count, a 128-byte frequency, a 128 byte raw-count, a count of comment lines, a ratio of number of comment lines to code lines, a number of code lines, a length of document in characters, a count of VBA scripts, an average number of lines per VBA script, a largest number of lines per VBA script;

receiving, from the machine learning-based classifier, malware classification data that indicates that the second electronic file includes malware; and

causing the network to modify, delay, or block transmission of the second electronic file to the second recipient computer.

7. The method of claim 1 , further comprising:

creating, by the pre-processor, a second output file based on input of a second electronic file, wherein the second output file indicates that the second electronic file is a Rich Text Format (RTF) file or a word processor-created document file that does not contain programming code, wherein the second electronic file is a second attachment to a second message that is in-transit to a second recipient computer on the network, and wherein the second output file includes features that are created by the pre-processor analyzing one or more sub-features of the second electronic file, wherein the features comprises any one or more of the following: a page count, a word count, a character count, an editing time, a classification, a warning, a text entropy, an overall entropy, a document size, parse tree data, a rule name, an indicator priority, an indicator category;

receiving, from the machine learning-based classifier, malware classification data that indicates that the second electronic file includes malware; and

causing the network to modify, delay, or block transmission of the second electronic file to the second recipient computer.

8. The method of claim 1 , further comprising:

creating, by the pre-processor, a second output file based on input of a second electronic file, wherein the second output file indicates that the second electronic file is a Portable Document Format (PDF) file or an image file, wherein the second electronic file is a second attachment to a second message that is in-transit to a second recipient computer on the network, and wherein the second output file includes features that are created by the pre-processor analyzing one or more sub-features of the second electronic file, wherein the features comprises any one or more of the following: a version number, a number of colors, an occurrence count for a string of interest, a count of embedded objects, a minimum object identifier, a maximum object identifier, a trailer minimum size, a trailer maximum size, a count of trailer objects, a count of streams, a stream minimum size, a stream maximum size, a fraction of a file that is part of a stream, a total entropy, a non-stream entropy, a file size in bytes, a file size in bytes for non-stream parts, an end of file (EOF) count, a count for a top occurring variable, a count of xref tags, a count of startxref tags, a count of embedded Uniform Resource Locators (URLs), a top occurring metadata item;

receiving, from the machine learning-based classifier, malware classification data that indicates that the second electronic file includes malware; and

causing the network to modify, delay, or block transmission of the second electronic file to the second recipient computer.

9. The method of claim 1 , further comprising:

creating, by the pre-processor, a second output file based on input of a second electronic file, wherein the second output file indicates that the second electronic file is a Flash file or a Small Web Format (SWF) file or a multimedia file, wherein the second electronic file is a second attachment to a second message that is in-transit to a second recipient computer on the network, and wherein the second output file includes features that are created by the pre-processor analyzing one or more sub-features of the second electronic file, wherein the features comprises any one or more of the following: a version number, a file size, a count of scripts, a count for a word of interest, a decompiler message, a line count, a character count, a 128 byte frequency, a 128 byte raw-count;

receiving, from the machine learning-based classifier, malware classification data that indicates that the second electronic file includes malware; and

causing the network to modify, delay, or block transmission of the second electronic file to the second recipient computer.

10. The method of claim 1 , comprising receiving the first electronic file from a message transfer application on the network.

11. A computer program product comprising:

one or more non-transitory computer-readable storage media comprising instructions which, when executed by one or more processors, cause the one or more processors to perform operations comprising:

receiving, from a pre-processor, a first output file;

wherein the first output file is created by the pre-processor in response to input of a first electronic file to the pre-processor, wherein the first output file indicates that the first electronic file is a Portable Document Format (PDF) file;

wherein the first electronic file is an attachment to a first message that is in-transit to a first recipient computer on a network;

wherein the first output file contains features that are created by the pre-processor analyzing one or more sub-features of the first electronic file and adjusting the features by one or more of: reducing the features using mutual information criteria and transforming the features to a floating point vector;

receiving, from a machine learning-based classifier, malware classification data that indicates whether the first electronic file does or does not contain malware;

wherein the malware classification data is output by the machine learning-based classifier in response to the machine learning-based classifier determining that the features are or are not indicators of obfuscation, and wherein the machine learning-based classifier is created using a neural network and the malware classification data is produced by inputting a single image of the first electronic file to the machine learning-based classifier;

wherein data used to create the machine learning-based classifier includes output files previously created by the pre-processor; and

in response to the malware classification data matching a criterion, causing the network to modify, delay, or block transmission of the first electronic file to the first recipient computer.

12. The computer program product of claim 11 , wherein the pre-processor performs any one or more of the following to create the first output file: parsing the first electronic file, pre-execution static analysis of the first electronic file, extracting script components from the first electronic file.

13. The computer program product of claim 11 , wherein the features comprises any one or more of the following: an occurrence count, an occurrence frequency, metadata, a string of interest, a warning, a classification, a rule name.

14. The computer program product of claim 11 , wherein the one or more sub-features of the first electronic file comprises any one or more of the following: a version number, an image, a video, a color, a string of interest, an embedded object, an object identifier, a trailer object, a stream, an end of file (EOF) indicator, a variable, an xref tag, a startxref tag, a Uniform Resource Locators (URL), a metadata item.

15. The computer program product of claim 11 , wherein a model used by the machine learning-based classifier is created using any one or more of the following: a tree-based algorithm, a random forest algorithm, a deep learning algorithm, a neural network, a deep convolutional neural network.

16. The computer program product of claim 11 , wherein the operations further comprise:

creating, by the pre-processor, a second output file based on input of a second electronic file, wherein the second output file indicates that the second electronic file contains Visual Basic for Application (VBA) code or programming code, wherein the second electronic file is a second attachment to a second message that is in-transit to a second recipient computer on the network, and wherein the second output file includes features that are created by the pre-processor analyzing one or more sub-features of the second electronic file, wherein the features comprises any one or more of the following: a word count, a 128-byte frequency, a 128 byte raw-count, a count of comment lines, a ratio of number of comment lines to code lines, a number of code lines, a length of document in characters, a count of VBA scripts, an average number of lines per VBA script, a largest number of lines per VBA script;

receiving, from the machine learning-based classifier, malware classification data that indicates that the second electronic file includes malware; and

causing the network to modify, delay, or block transmission of the second electronic file to the second recipient computer.

17. The computer program product of claim 11 , wherein the operations further comprise:

creating, by the pre-processor, a second output file based on input of a second electronic file, wherein the second output file indicates that the second electronic file is a Rich Text Format (RTF) file or a word processor-created document file that does not contain programming code, wherein the second electronic file is a second attachment to a second message that is in-transit to a second recipient computer on the network, and wherein the second output file includes features that are created by the pre-processor analyzing one or more sub-features of the second electronic file, wherein the features comprises any one or more of the following: a page count, a word count, a character count, an editing time, a classification, a warning, a text entropy, an overall entropy, a document size, parse tree data, a rule name, an indicator priority, an indicator category;

receiving, from the machine learning-based classifier, malware classification data that indicates that the second electronic file includes malware; and

causing the network to modify, delay, or block transmission of the second electronic file to the second recipient computer.

18. The computer program product of claim 11 , wherein the operations further comprise:

creating, by the pre-processor, a second output file based on input of a second electronic file, wherein the second output file indicates that the second electronic file is a Portable Document Format (PDF) file or an image file, wherein the second electronic file is a second attachment to a second message that is in-transit to a second recipient computer on the network, and wherein the second output file includes features that are created by the pre-processor analyzing one or more sub-features of the second electronic file, wherein the features comprises any one or more of the following: a version number, a number of colors, an occurrence count for a string of interest, a count of embedded objects, a minimum object identifier, a maximum object identifier, a trailer minimum size, a trailer maximum size, a count of trailer objects, a count of streams, a stream minimum size, a stream maximum size, a fraction of a file that is part of a stream, a total entropy, a non-stream entropy, a file size in bytes, a file size in bytes for non-stream parts, an end of file (EOF) count, a count for a top occurring variable, a count of xref tags, a count of startxref tags, a count of embedded Uniform Resource Locators (URLs), a top occurring metadata item;

receiving, from the machine learning-based classifier, malware classification data that indicates that the second electronic file includes malware; and

causing the network to modify, delay, or block transmission of the second electronic file to the second recipient computer.

19. The computer program product of claim 11 , wherein the operations further comprise:

creating, by the pre-processor, a second output file based on input of a second electronic file, wherein the second output file indicates that the second electronic file is a Flash file or a Small Web Format (SWF) file or a multimedia file, wherein the second electronic file is a second attachment to a second message that is in-transit to a second recipient computer on the network, and wherein the second output file includes features that are created by the pre-processor analyzing one or more sub-features of the second electronic file, wherein the features comprises any one or more of the following: a version number, a file size, a count of scripts, a count for a word of interest, a decompiler message, a line count, a character count, a 128 byte frequency, a 128 byte raw-count;

receiving, from the machine learning-based classifier, malware classification data that indicates that the second electronic file includes malware; and

causing the network to modify, delay, or block transmission of the second electronic file to the second recipient computer.

20. The computer program product of claim 11 , the instructions, when executed by one or more processors, cause the one or more processors to perform operations comprising receiving the first electronic file from a message transfer application on the network.