Automated program repair tool

1. A computer-implemented method, comprising:

pre-training a neural transformer model with attention with an unsupervised training dataset, the unsupervised training dataset including a plurality of sequences of source code;

fine-tuning the neural transformer model with attention with a supervised training dataset, the supervised training dataset based a triplet including a code snippet with a bug, a code repair for the bug, and an annotated bug type;

obtaining a first code snippet having an identified bug type and a bug-type edit centroid associated with the identified bug type, wherein the bug-type edit centroid represents edits to fix the annotated bug type; and

applying the neural transformer model with attention to generate a first code repair for the first code snippet given the first source code snippet, the identified bug type, and the bug-type edit centroid.

2. The method of claim 1 , further comprising:

applying a span masking function to each sequence of source code to mask out a subset of subtokens in a sequence; and

wherein the neural transformer model with attention learns original subtokens of the sequence.

3. The method of claim 1 , wherein fine-tuning the neural transformer model with attention with a supervised training dataset further comprises:

generating a bug edit embedding representing edits made to correct a bug; and

computing a bug-type edit centroid for each bug type from the bug edit embeddings of a particular bug type.

4. The method of claim 3 , wherein the neural transformer model with attention includes one or more encoder blocks coupled to one or more decoder blocks.

5. The method of claim 4 , wherein fine-tuning the neural transformer model with attention with supervised training dataset further comprises:

concatenating the bug edit embedding with output from a last encoder block to input to a first decoder block or to encoder-decoder attention block, and concatenating the bug edit embedding with output embedding at each temporal step.

6. The method of claim 1 , further comprising:

identifying the annotated bug type through an interprocedural static analysis of the code snippet.

7. The method of claim 1 , wherein the neural transformer model with attention includes one or more encoder blocks and one or more decoder blocks, wherein an encoder block contains a multi-head attention layer and a feed-forward neural network, wherein a decoder block contains a masked multi-head attention layer, an encoder-decoder multi-head attention layer, and a feed-forward neural network.

8. The method of claim 1 , wherein the annotated bug type includes a null pointer dereference, a memory leak, an immutable cast, empty vector access, or thread safety violation.

9. A system comprising:

one or more processors; and

a memory that stores one or more programs that are configured to be executed by the one or more processors, the one or more programs including instructions to perform actions that:

obtain a code snippet with a source code bug;

identify an annotated bug type for the source code bug from an interprocedural static code analysis of the code snippet;

obtain a bug-type edit centroid of the annotated bug type, wherein the bug-type edit centroid represents edits to fix the annotated bug type;

predict a bug repair for the code snippet from a neural transformer model with attention given the code snippet, the annotated bug type, and the bug-type edit centroid; and

utilize the predicted bug repair to repair the code snippet.

10. The system of claim 9 , wherein the annotated bug type includes a null dereference, an immutable cast, an empty vector access, a memory leak, or a thread-safety violation.

11. The system of claim 9 , wherein the one or more programs include further instructions to perform actions that:

pre-train the neural transformer model with attention with an unsupervised training dataset, the unsupervised training dataset including source code snippets.

12. The system of claim 11 , wherein the one or more programs include further instructions to perform actions that:

fine-tune the pre-trained neural transformer model with attention with a supervised training dataset, the supervised training dataset containing translation tasks, a translation task containing a source code with a bug, a bug type annotation of the bug, and a bug fix for the bug.

13. The system of claim 12 , wherein the one or more programs include further instructions to perform actions that:

generate a bug edit representation for each bug within the supervised training dataset; and

compute a bug-type edit centroid for each bug type based on the bug edit representations of each bug type.

14. The system of claim 13 , wherein the neural transformer model with attention includes one or more encoder blocks and one or more decoder blocks.

15. The system of claim 14 , wherein the one or more programs include further instructions to perform actions that:

utilize the bug edit representations in the one or more decoder blocks during a training stage, and utilize bug-type edit centroid in the one or more decoder blocks during inference stage, the bug-type edit centroid of a same bug type as the annotated bug type.

16. A device, comprising:

at least one processor and a memory;

wherein the at least one processor is configured to perform actions that:

train a neural transformer model with attention to learn to translate a source code snippet with a bug, bug type, and bug edit representation for the bug type into a code snippet with a repair for the bug by transfer learning,

wherein the transfer learning pre-trains the neural transformer model with attention from a plurality of unsupervised training data, the plurality of unsupervised training data including code snippets from a plurality of source code programs,

wherein the transfer learning fine-tunes the pre-trained neural transformer model with attention using a plurality of translation tasks, a translation task including a triplet having a code snippet with a bug, a code snippet with a repair for the bug, and a bug type for the bug; and

utilize the neural transformer model with attention to predict a code repair for a second code snippet having a bug, a bug type, and a bug-type edit centroid of the bug type, wherein the bug-type edit centroid represents edits of the annotated bug type.

17. The device of claim 16 , wherein the at least one processor is further configured to perform actions that:

utilize a static code analyzer to identify the bug type of the second code snippet.

18. The device of claim 16 , wherein the at least one processor is further configured to perform actions that:

generate a bug edit representation for each translation task; and

compute the bug-type edit centroid for each bug type based on each bug edit representation of a bug type.

19. The device of claim 18 , wherein the neural transformer model with attention includes one or more encoder blocks coupled to one or more decoder blocks, wherein output of a last encoder block is input into each of the decoder blocks.

20. The device of claim 19 , wherein the at least one processor is further configured to perform actions that:

concatenate the output of the last encoder block with a bug type centroid of a bug type of a fine-tuning triplet to a first decoder block.