Recurrent neural networks for malware analysis
Using a recurrent neural network (RNN) that has been trained to a satisfactory level of performance, highly discriminative features can be extracted by running a sample through the RNN, and then extracting a final hidden state h i , where i is the number of instructions of the sample. This resulting feature vector may then be concatenated with the other hand-engineered features, and a larger classifier may then be trained on hand-engineered as well as automatically determined features. Related apparatus, systems, techniques and articles are also described.
1. A system comprising:
computer hardware configured to perform operations comprising:
feeding data encapsulating a sample of at least a portion of one or more files into a recurrent neural network trained using historical data;
extracting, by the RNN, a plurality of final hidden states in a hidden layer of the recurrent neural network; and
determining, using the recurrent neural network and the plurality of final hidden states, whether at least a portion of the sample comprises malicious code;
wherein:
the recurrent neural network comprises an Elman network that parameterizes a function ƒ(x, ht−1) as ht=g(W1x+Rht−1);
where the hidden state h t comprises a time-dependent function of the input x as well as a previous hidden state ht−1, W 1 is a matrix defining input-to-hidden connections, R is a matrix defining the recurrent connections, and g(⋅) is a differentiable nonlinearity.
2. The system of claim 1 , wherein the operations further comprise:
receiving and/or accessing the data as at least part of a data stream.
3. The system of claim 1 , wherein the sample comprises a series of fixed-length encoded words.
4. The system of claim 1 , wherein the sample comprises a series of instructions.
5. The system of claim 1 , wherein a hidden state at a time t(h t ) is defined by: ht=ƒ(x,ht−1), and wherein the hidden state ht comprises a time-dependent function of an input x as well as a previous hidden state ht−1.
6. The system of claim 1 , wherein the Elman network comprises deep transition or decoding functions.
7. The system of claim 1 , wherein the operations further comprise: adding an output layer on top of at least one hidden layer of the plurality of hidden layers such that σt=σ(W2ht) where o t is an output, W 2 defines a linear transformation of hidden activations, and a σ(⋅) is a logistic function.
8. The system of claim 7 , wherein the operations further comprise:
applying backpropagation through time by which W 2 , W 1 , and R are iteratively refined to drive the output of to a desired value as portions of the sample are passed through the RNN.
9. The system of claim 1 , wherein the operations further comprise:
providing a characterization of the sample as malicious or not malicious based at least in part on the determining.
10. The system of claim 1 , wherein the providing of the characterization comprises at least one of: transmitting the data to a remote computing system, loading the data into memory, or storing the data.
11. The system of claim 1 , wherein the one or more files comprise binary files and/or executable files.
12. The system of claim 1 , wherein the computer hardware comprises a programmable data processor and a memory storing instructions that when executed by the at least one programmable data processor, result in at least some of the operations.
13. A method for implementation by at least one computing device comprising:
feeding data encapsulating a sample of at least a portion of one or more files into a recurrent neural network trained using historical data;
extracting, by the RNN, a plurality of final hidden states in a hidden layer of the recurrent neural network; and
determining, using the recurrent neural network and the plurality of final hidden states, whether at least a portion of the sample to comprises malicious code;
wherein:
the recurrent neural network comprises an Elman network that parameterizes a function ƒ(x, ht−1) as ht=g(W1x+Rht−1);
where the hidden state h t comprises a time-dependent function of the input x as well as a previous hidden state ht−1, W 1 is a matrix defining input-to-hidden connections, R is a matrix defining the recurrent connections, and g(⋅) is a differentiable nonlinearity.
14. The method of claim 13 further comprising:
receiving and/or accessing the data as at least part of a data stream.
15. The method of claim 13 , wherein the sample comprises a series of fixed-length encoded words and/or a series of instructions.
16. The method of claim 13 , wherein a hidden state at a time t(h t ) is defined by: ht=ƒ(x, ht−1), and wherein the hidden state ht comprises a time-dependent function of an input x as well as a previous hidden state ht−1.
17. The method of claim 13 , wherein the Elman network comprises deep transition or decoding functions.
18. The method of claim 13 further comprising:
adding an output layer on top of at least one hidden layer of the plurality of hidden layers such that σt=σ(W2ht) where of is an output, W 2 defines a linear transformation of hidden activations, and a σ(⋅) is a logistic function; and
applying backpropagation through time by which W 2 , W 1 , and R are iteratively refined to drive the output of to a desired value as portions of the sample are passed through the RNN.
19. The method of claim 13 further comprising:
providing a characterization of the sample as malicious or not malicious based at least in part on the determining;
wherein the providing of the characterization comprises at least one of: transmitting the data to a remote computing system, loading the data into memory, or storing the data;
wherein the one or more files comprise binary files and/or executable files.
20. A non-transitory computer program product storing instructions which, when executed by at least one programmable data processor forming part of at least one computing device, result in operations comprising:
feeding data encapsulating a sample of at least a portion of one or more files into a recurrent neural network trained using historical data;
extracting, by the RNN, a plurality of final hidden states in a hidden layer of the recurrent neural network; and
determining, using the recurrent neural network and the plurality of final hidden states, whether at least a portion of the sample is comprises malicious code;
wherein:
the recurrent neural network comprises an Elman network that parameterizes a function ƒ(x, ht−1) as ht=g(W1x+Rht−1);
where the hidden state h t comprises a time-dependent function of the input x as well as a previous hidden state ht−1, W 1 is a matrix defining input-to-hidden connections, R is a matrix defining the recurrent connections, and g(⋅) is a differentiable nonlinearity.