Multi-lingual code generation with zero-shot inference
A neural transformer model with attention is trained to predict candidates to complete a line of source code with a zero-inference capability. The model is trained on an unsupervised training dataset that includes features from source code written in multiple programming languages. The features include a file-level context and a local context, where the file-level context includes a global context, a class context, a function context, and/or a method context for each class, function and/or method of the source code programs used in the training dataset. The local context includes method bodies, function bodies, and/or stand-alone code of main method routines. From these features, the model is able to learn to predict an ordered sequence of code elements that complete a line of source code in a programming language seen and not seen during training.
1 . A method for automatic code completion, the method comprising:
obtaining a file-level context and a local context of a source code program;
generating a source code candidate by applying a trained deep learning model given the file-level context and the local context;
providing the source code candidate to complete a partially formed line of source code of the source code program and form a completed line of source code; and
writing the completed line of source code in the source code program written in a first programming language,
wherein the generating a source code candidate by applying a trained deep learning model further comprises:
transforming, utilizing a neural transformer model, a sequence of tokens into a byte-pair encoding to generate a plurality of partial source code candidates.
2 . The method of claim 1 , wherein the trained deep learning model comprises the neural transformer model with attention,
wherein the deep learning model has been trained on a training dataset that include a file context and a local context from a training source code program written in a second programming language, the second programming language is different from the first programming language.
3 . The method of claim 2 , wherein the generating a source code candidate by applying a trained deep learning model further comprises:
generating the sequence of tokens representing the file-level context and the local context.
4 . The method of claim 2 , wherein the neural transformer model with attention is trained with training datasets from a plurality of programming languages.
5 . The method of claim 1 , wherein the file-level context includes a global context, a method context, a class context, or a function context.
6 . The method of claim 1 , wherein the local context comprises a context of the source code program preceding the partially formed line of source code of the source code program.
7 . The method of claim 1 , wherein the local context comprises a method body, function body or stand-alone code of a main method.
8 . A code completion system comprising:
one or more processors; and
a memory that stores one or more programs that are configured to be executed by the one or more processors, the one or more programs including instructions to perform actions comprising:
obtaining a file-level context and a local context of a source code program;
generating a source code candidate by applying a trained deep learning model given the file-level context and the local context;
providing the source code candidate to complete a partially formed line of source code of the source code program and form a completed line of source code; and
writing the completed line of source code in the source code program written in a first programming language,
wherein the generating a source code candidate by applying a trained deep learning model further comprises:
transforming, utilizing a neural transformer model, a sequence of tokens into a byte-pair encoding to generate a plurality of partial source code candidates.
9 . The system of claim 8 , wherein the trained deep learning model comprises the neural transformer model with attention,
wherein the deep learning model has been trained on a training dataset that include a file context and a local context from a training source code program written in a second programming language, the second programming language is different from the first programming language.
10 . The system of claim 9 , wherein the operations further comprise:
generating the sequence of tokens representing the file-level context and the local context.
11 . The system of claim 9 , wherein the neural transformer model with attention is trained with training datasets from a plurality of programming languages.
12 . The system of claim 8 , wherein the file-level context includes a global context, a method context, a class context, or a function context.
13 . The system of claim 8 , wherein the local context comprises a context of the source code program preceding the partially formed line of source code of the source code program.
14 . The system of claim 8 , wherein the local context comprises a method body, function body or stand-alone code of a main method.
15 . A non-transitory computer-readable medium including instructions, which when executed by a processor, cause the processor to perform operations comprising:
obtaining a file-level context and a local context of a source code program;
generating a source code candidate by applying a trained deep learning model given the file-level context and the local context; and
providing the source code candidate to complete a partially formed line of source code of the source code program and form a completed line of source code; and
writing the completed line of source code in the source code program written in a first programming language, wherein the generating a source code candidate by applying a trained deep learning model further comprises:
transforming, utilizing a neural transformer model, a sequence of tokens into a byte-pair encoding to generate a plurality of partial source code candidates.
16 . The non-transitory computer-readable medium of claim 15 , wherein the trained deep learning model comprises a neural transformer model with attention,
wherein the deep learning model has been trained on a training dataset that include a file context and a local context from a training source code program written in a second programming language, the second programming language is different from the first programming language.
17 . The non-transitory computer-readable medium of claim 16 , wherein the operations further comprise:
generating the sequence of tokens representing the file-level context and the local context.
18 . The non-transitory computer-readable medium of claim 16 , wherein the neural transformer model with attention is trained with training datasets from a plurality of programming languages.
19 . The non-transitory computer-readable medium of claim 15 , wherein the file-level context includes a global context, a method context, a class context, or a function context.
20 . The non-transitory computer-readable medium of claim 15 , wherein the local context comprises a context of the source code program preceding the partially formed line of source code of the source code program.