Transformer-based bug fixing
Techniques for determining buggy code are described. An encoder/decoder-based (e.g., transformer-based) model approach is described. In some embodiments, a service receives request to perform transformer-based bug fixing on code, performs bug fixing inference to the code by applying a trained encoder/decoder-based model, and reports out a result of the inference, wherein the output includes an indication of a location of a potential edit to be made in the code and the potential edit in the code.
1. A computer-implemented method comprising:
receiving a request to perform transformer-based bug fixing on code;
performing bug fixing inference on the code by applying a transformer model, wherein the transformer model includes an encoder to generate an encoder memory from tokens from the code and a decoder to predict a next token in an editing sequence, wherein the decoder includes a selectable first branch to predict edit actions and inserted words and a selectable second branch that uses a pointer network to predict edit locations based at least partially on the encoder memory generated by the encoder of the transformer model, wherein the bug fixing inference is performed using a beam search over output of the encoder, and wherein the beam search uses a finite state machine to determine a branch of the decoder to use; and
reporting out a result of the bug fixing inference, wherein the result includes an indication of a location of a potential edit to be made in the code and the potential edit to be made in the code.
2. The computer-implemented method of claim 1 , further comprising reranking output of the beam search using a trained reranker to generate a new ordering of editing sequences.
3. A computer-implemented method comprising:
receiving a request to perform encoder/decoder-based bug fixing on code;
performing bug fixing inference on the code by applying a trained encoder/decoder-based model, wherein the model includes an encoder to generate an encoder memory from tokens from the code and a decoder to predict a next token in an editing sequence, wherein the decoder includes a selectable first branch to predict edit actions and inserted words and a selectable second branch to predict edit locations based at least partially on the encoder memory generated by the encoder of the model, wherein the bug fixing inference is performed using a beam search over output of the encoder, and wherein the beam search uses a finite state machine to determine a branch of the decoder to use; and
reporting out a result of the bug fixing inference, wherein the result includes an indication of a location of a potential edit to be made in the code and the potential edit to be made in the code.
4. The computer-implemented method of claim 3 , further comprising reranking output of the model to select one or more hypotheses produced by the beam search using a reranker.
5. The computer-implemented method of claim 4 , wherein the reranker is fed output of the encoder that has been fed edited code and buggy code.
6. The computer-implemented method of claim 4 , wherein the reranker is fed output of the decoder that has been fed an edit sequence and the encoder memory.
7. The computer-implemented method of claim 4 , wherein the reranking comprises reranking the output of the model using an ensemble model that blends a plurality of reranker outputs.
8. The computer-implemented method of claim 4 , wherein the reranker determines if a predicted edit is adequate to produce as an output based on a comparison to a threshold.
9. The computer-implemented method of claim 3 , wherein the finite state machine uses a grammar that includes delete and insert actions, and wherein a delete action deletes an indexed subsequence and an insert action inserts a sequence of tokens.
10. The computer-implemented method of claim 3 , wherein the second branch uses a pointer network to predict the edit locations.
11. The computer-implemented method of claim 3 , further comprising tokenizing the code prior to applying the model.
12. The computer-implemented method of claim 3 , wherein the request to perform transformer-based bug fixing on code includes one or more of the code to evaluate for bug fixing, a location of the code to evaluate, or an indication of how to provide the result of the bug fixing inference.
13. The computer-implemented method of claim 3 , wherein the code to evaluate is unnormalized code.
14. A system comprising:
a first one or more electronic devices to implement a code repository service in a multi-tenant provider network; and
a second one or more electronic devices to implement a code recommendation service in the multi-tenant provider network, the code recommendation service including instructions that upon execution by one or more processors cause the code recommendation service to:
receive a request to perform transformer-based bug fixing on code stored in the code repository service;
on the code by applying a trained encoder/decoder-based model, wherein the model includes an encoder to generate an encoder memory from tokens from the code and a decoder to predict a next token in an editing sequence, wherein the decoder includes a selectable first branch to predict edit actions and inserted words and a selectable second branch to predict edit locations based at least partially on the encoder memory generated by the encoder of the model, wherein the bug fixing inference is performed using a beam search over output of the encoder, and wherein the beam search uses a finite state machine to determine a branch of the decoder to use; and
report out a result of the bug fixing inference, wherein the result includes an indication of a location of a potential edit to be made in the code and the potential edit to be made in the code.
15. The system of claim 14 , wherein the request to perform transformer-based bug fixing on code includes one or more of the code to evaluate for bug fixing, a location of the code to evaluate, or an indication of how to provide the result of the bug fixing inference.
16. The system of claim 14 , wherein the code recommendation service is further to rerank output of the model to select one or more hypotheses produced by the beam search using a reranker.
17. The system of claim 16 , wherein the reranker is fed output of the encoder that has been fed edited code and buggy code.
18. The system of claim 16 , wherein the reranker is fed output of the decoder that has been fed an edit sequence and the encoder memory.
19. The system of claim 16 , wherein the reranking comprises reranking the output of the model using an ensemble model that blends a plurality of reranker outputs.
20. The system of claim 16 , wherein the reranker determines if a predicted edit is adequate to produce as an output based on a comparison to a threshold.