Methods and apparatus for machine learning based malware detection
Apparatus and methods describe herein, for example, a process that can include receiving a potentially malicious file, and dividing the potentially malicious file into a set of byte windows. The process can include calculating at least one attribute associated with each byte window from the set of byte windows for the potentially malicious file. In such an instance, the at least one attribute is not dependent on an order of bytes in the potentially malicious file. The process can further include identifying a probability that the potentially malicious file is malicious, based at least in part on the at least one attribute and a trained threat model.
1. A method, comprising:
receiving a target file;
dividing the target file into a plurality of byte windows;
for each byte window from the plurality of byte windows, determining (1) how often a byte value in that byte window is within a byte value range and (2) whether an informational entropy for that byte window is within an informational entropy range;
calculating a probability that the target file is malicious based on the determining;
communicating a threat score based on the probability, the threat score corresponding to an indication of a potential threat to a user, device, or network; and
performing a remedial action based on the threat score.
2. An apparatus, comprising:
a memory; and
a hardware processor operatively coupled to the memory,
the hardware processor configured to receive a target file, the hardware processor configured to calculate an attribute associated with the target file, the attribute based on, for each byte window from a plurality of byte windows of the target file, how often a byte value within a byte value range is within that byte window and whether an informational entropy associated with that byte window is within an informational entropy range,
the hardware processor further configured to calculate, based on the attribute, a probability that the target file is malicious, and output a threat score based on the probability.
3. The apparatus of claim 2 , wherein the threat score corresponds to a value for indicating a potential threat to a user or a device.
4. The apparatus of claim 2 , wherein the threat score corresponds to a value for indicating a potential threat to a network.
5. The apparatus of claim 2 , wherein the threat score corresponds to a threat indication for the target file based on the probability.
6. The apparatus of claim 2 , wherein the hardware processor is further configured to calculate the attribute using a trained threat model.
7. The apparatus of claim 6 , wherein the trained threat model includes at east one of a random forest classifier or a deep neural network classifier.
8. The apparatus of claim 2 , wherein the hardware processor is further configured to calculate the threat score based on a security indication associated with a network.
9. The apparatus of claim 2 , wherein the hardware processor is further configured to calculate the threat score based on a rate of false positives associated with a trained threat model.
10. The apparatus of claim 2 , wherein the hardware processor is further configured to calculate the threat score based on a rate of false negatives associated with a trained threat model.
11. The apparatus of claim 2 , wherein the hardware processor is configured to calculate the threat score using metadata associated with the target file.
12. The apparatus of claim 2 , wherein the attribute is also based on how often a combination of the informational entropy range and a byte standard deviation range is associated with the plurality of byte windows of the target file.
13. The apparatus of claim 2 , wherein the attribute is also based on an indication of how often a combination of a string length range and a string hash value range occurs in each byte window from the plurality of byte windows.
14. A non-transitory processor-readable medium storing code representing instructions to be executed by a processor, the instructions comprising code to cause the processor to:
receive a target file;
determine an attribute associated with the target file, the attribute based on, for each byte window from a plurality of byte windows of the target file, how often a byte value within a byte value range is within that byte window, and whether an informational entropy associated with that byte window is within an informational entropy range;
determine a probability that the target file is malicious based on the attribute; and
output a threat score based on the probability.
15. The non-transitory processor-readable medium of claim 14 , wherein the threat score corresponds to a value for indicating a potential threat to a user, a device, or a network.
16. The non-transitory processor-readable medium of claim 14 , wherein the code to cause the
processor to determine the probability includes code to cause the processor to determine the probability that the target file is malicious using a trained threat model, the trained threat model including at least one of a random forest classifier or a deep neural network classifier.
17. The non-transitory processor-readable medium of claim 14 , wherein the code to cause the processor to determine the probability includes code to cause the processor to determine the probability using at least one of a rate of false positives associated with a trained threat model, or a rate of false negatives associated with the trained threat model.
18. The non-transitory processor-readable medium of claim 14 , further comprising code to cause the processor to:
calculate a byte standard deviation value for each byte window from the plurality of byte windows of the target file, the code to cause the processor to determine the attribute includes code to cause the processor to determine the attribute based on comparing the byte standard deviation value of each byte window from the plurality of byte windows to a predefined range.
19. The non-transitory processor-readable medium of claim 14 , wherein the code to cause the processor to determine the probability includes code to cause the processor to determine the probability that the target file is malicious using metadata associated with the target file.
20. The non-transitory processor-readable medium of claim 14 , wherein the target file is a Portable Executable (PE) file and the probability is further determined based on information identified from Portable Executable (PE) fields associated with the PE file.