Multi-lingual code generation with zero-shot inference

1. A system comprising:

one or more processors; and

a memory that stores one or more programs that are configured to be executed by the one or more processors, the one or more programs including instructions to perform actions that:

obtain a source code program in an edit session;

extract a file-level context of the source code program;

extract a local context of the source code program at a current cursor position;

generate a candidate to complete a line of source code of the source code program, the source code program written in a first programming language, the candidate generated by a deep learning model given the file-level context and the local context, wherein the deep learning model is trained to learn syntax of a plurality of different programming languages, wherein the plurality of different programming languages differs from the first programming language; and

provide the candidate to the source code program.

2. The system of claim 1 , wherein deep learning model is a decoder neural transformer model with attention.

3. The system of claim 1 , wherein the file-level context includes a global context, a method context, a class context and/or a function context.

4. The system of claim 1 , wherein the local context includes a context of the source code program immediately preceding the current cursor position.

5. The system of claim 1 , wherein the local context includes a method body, function body or stand-alone code of a main method.

6. The system of claim 1 , wherein the one or more programs include further instructions that:

generate a sequence of subtokens of length T representing the file-level context and the local context, the file-level context including T/4 subtokens and the local context including 3T/4 subtokens; and

input the sequence of subtokens of length T to the deep learning model to generate the candidate.

7. The system of claim 6 , wherein the one or more programs include further instructions that:

transform the sequence of subtokens of length T into a byte-level byte-pair encoding.

8. A method, comprising:

generating an ordered sequence of subtokens representing a context of a source code program in a source code editor at a current cursor position, the source code program written in a first programming language, the context including a file-level context and a local context;

processing the ordered sequence of subtokens by a deep learning model to generate a candidate to complete a line of source code at the current cursor position,

wherein the deep learning model is trained to recognize source code written in a second programming language, wherein the first programming language and the second programming language differ,

wherein k partial candidate sequences are generated at each iteration of the processing using the deep learning model to predict a next subtoken for a partial candidate sequence by generating an output probability distribution of the subtokens at each iteration,

wherein the processing selects top k subtokens having highest probability to complete a partial candidate sequence,

wherein the processing ceases when the deep learning model predicts an end-of-line token as next subtoken for a select partial candidate sequence; and

providing the select partial candidate sequence as a candidate to complete the line of source code at the current cursor position.

9. The method of claim 8 , comprising:

forming the ordered sequence of subtokens of length T including subtokens representing the file-level context and subtokens representing the local context.

10. The method of claim 9 , wherein the ordered sequence of subtokens of length T include a first sequence of subtokens of length T/4 representing the file-level context and a second sequence of subtokens of length 3T/4 representing the local context.

11. The method of claim 8 , wherein the file-level context includes at least one of a global context, a class context, a method context or a function context.

12. The method of claim 8 , wherein the local context includes a method body, a function body, or stand-alone code associated with a main routine.

13. The method of claim 8 , wherein the deep learning model is a decoder neural transformer with attention.

14. The method of claim 9 , further comprising:

transforming the sequence of subtokens of length T into a byte-level byte-pair encoding.

15. A device, comprising:

at least one processor coupled to a memory device;

wherein the at least one processor is configured to perform acts to:

obtain a deep learning model trained to predict a candidate to complete a line of source code of a source code program in a source code editor, wherein the deep learning model is trained on an unsupervised training dataset of features from source code programs written in multiple different programming languages; and

deploy the deep learning model to predict a candidate to complete a line of source code at a current cursor position in a first source code program given a file context and a local context of the first source code program, the local context extracted from the current cursor position, the first source code program written in a programming language outside of the multiple different programming languages.

16. The device of claim 15 , wherein the deep learning model is a decoder neural transformer with attention.

17. The device of claim 15 , wherein the file-level context includes a global context, a class context, function context, and/or a method context of the source code program.

18. The device of claim 15 , wherein the local context includes a method body, function body or stand-alone code of a main routine.