Deep learning model for learning program embeddings

1. A method comprising:

receiving a plurality of execution traces of a program, each execution trace comprising a plurality of variable values, and wherein each execution trace of the program includes a semantic label describing a processing function of the program;

encoding, by a first recurrent neural network, the plurality of variable values to generate a plurality of program states for each execution trace;

determining, by a bi-directional recurrent neural network, a reduced set of program states for each execution trace from the plurality of program states,

wherein determining the reduced set of program states for each execution trace from the plurality of program states comprises:

computing a forward sequence from the plurality of program states;

computing a backward sequence from the plurality of program states;

for each program state in the plurality of program states:

determining a forward context vector from the forward sequence;

determining a backward context vector from the backward sequence;

applying a multi-layer perceptron to predict a necessity of each program state;

and

determining, based on a result of the multi-layer perceptron,

whether to include the program state in the reduced set of program states;

encoding, by a second recurrent neural network, the reduced set of program states to generate a plurality of executions for the program;

pooling the plurality of executions to generate a program embedding;

predicting semantics of the program using the program embedding in order to classify the program, wherein the predicted semantics include one or more semantic labels; and

using the predicted semantics of the program to perform a compiler optimization task, wherein the method is optimized to minimize a cross-entropy loss and minimize a number of program states in the reduced set of program states.

2. The method of claim 1 , wherein the predicting semantics of the program comprises returning a probability of one of a list of program functionalities.

3. The method of claim 1 , wherein there are 50% fewer states in the reduced set of program states than in the plurality of program states.

4. The method of claim 1 , wherein each execution trace in the plurality of execution traces is generated with randomly generated inputs.

5. The method of claim 1 , wherein determining the forward context vector comprises pooling a subset of components of the forward sequence, the subset of components of the forward sequence comprising information from a first s-1 program states, and wherein determining the backward context vector comprises pooling a subset of components of the backward sequence, the subset of components of the backward sequence comprising information from a last m-s program states.

6. The method of claim 1 , wherein the predicting semantics is performed with an output of a softmax function.

7. The method of claim 1 , wherein the program has been mutated.

8. A computer comprising:

a processor; and

a computer-readable medium comprising code, executable by the processor, for implementing a method comprising:

receiving a plurality of execution traces of a program, each execution trace comprising a plurality of variable values, and wherein each execution trace of the program includes a semantic label describing a processing function of the program;

encoding, by a first recurrent neural network, the plurality of variable values to generate a plurality of program states for each execution trace;

determining, by a bi-directional recurrent neural network, a reduced set of program states for each execution trace from the plurality of program states,

wherein determining the reduced set of program states for each execution trace from the plurality of program states comprises:

computing a forward sequence from the plurality of program states;

computing a backward sequence from the plurality of program states;

for each program state in the plurality of program states:

determining a forward context vector from the forward sequence;

determining a backward context vector from the backward sequence;

applying a multi-layer perceptron to predict a necessity of each program state; and

determining, based on a result of the multi-layer perceptron,

whether to include the program state in the reduced set of program states;

encoding, by a second recurrent neural network, the reduced set of program states to generate a plurality of executions for the program;

pooling the plurality of executions to generate a program embedding;

predicting semantics of the program using the program embedding in order to classify the program, wherein the predicted semantics include one or more semantic labels; and

using the predicted semantics of the program to perform a compiler optimization task, wherein the method is optimized to minimize a cross-entropy loss and minimize a number of program states in the reduced set of program states.

9. The computer of claim 8 , wherein the predicting semantics of the program comprises returning a probability of one of a list of program functionalities.

10. The computer of claim 8 , wherein there are 50% fewer states in the reduced set of program states than in the plurality of program states.

11. The computer of claim 8 , wherein each execution trace in the plurality of execution traces is generated with randomly generated inputs.

12. The computer of claim 8 , wherein determining the forward context vector comprises pooling a subset of components of the forward sequence, the subset of components of the forward sequence comprising information from a first s-1 program states, and wherein determining the backward context vector comprises pooling a subset of components of the backward sequence, the subset of components of the backward sequence comprising information from a last m-s program states.

13. The computer of claim 8 , wherein the predicting semantics is performed with an output of a softmax function.

14. The computer of claim 8 , wherein the program has been mutated.

15. The method according to claim 1 , wherein a deep learning model predicts the semantics of the program, and the method further comprises evaluating an accuracy of tasks performed by the deep learning model.