Neural transformer code completion for command line interface
A code completion system for a CLI utilizes neural transformer models with attention to generate candidates to complete a line of CLI code. The code completion system uses a first deep learning model to predict at most k candidate command names to follow n immediately preceding lines of CLI code which are presented to a developer. Upon the developer accepting one of the candidate command names, the code completion system uses a second deep learning model to predict at most k parameter strings to complete the line of CLI code.
1 . A system comprising:
one or more processors; and
a memory that stores one or more programs that are configured to be executed by the one or more processors, the one or more programs including instructions to perform actions to:
track lines of command line interface (CLI) code entered into a user interface, wherein a line of CLI code comprises a command name and a parameter string, wherein the parameter string comprises including zero or one or more parameters, wherein a parameter comprises a parameter value;
acquire n consecutive lines of CLI code from the user interface immediately preceding a current position in the user interface;
cause a first deep learning model to generate k model-generated command names given the n consecutive lines of CLI code;
present the k model-generated command names in the user interface;
retrieve a user-selected one of the k model-generated command names;
cause a second deep learning model to generate a parameter string given the user-selected command name and the n consecutive lines of CLI code, wherein the parameter string completes the user-selected command name into a syntactically-correct line of CLI code, wherein the first deep learning model and the second deep learning model differ;
output the user-selected command name and the model-generated parameter string as a syntactically-correct line of CLI code; and
upon user selection of the syntactically-correct line of CLI code, input the syntactically-correct line of CLI code into the CLI.
2 . The system of claim 1 , wherein the syntactically-correct line of CLI code spans several physical lines in the CLI.
3 . The system of claim 1 , wherein the n consecutive lines of CLI code are based on a number of lines of CLI code used to train the first deep learning model and/or the second deep learning model.
4 . The system of claim 1 , wherein the first deep learning model is based on a pre-trained encoder model having been trained on an unsupervised training dataset of source code programs from different programming languages using a masked language objective and fine-tuned on a supervised training dataset of CLI code snippets.
5 . The system of claim 1 , wherein the second deep learning model is based on a pre-trained decoder model having been trained on an unsupervised training dataset of source code programs from different programming languages and fine-tuned on a supervised training dataset of CLI code snippets.
6 . The system of claim 1 , wherein the first deep learning model is an encoder-only neural transformer model with attention.
7 . The system of claim 1 , wherein the second deep learning model is a decoder-only neural transformer model with attention.
8 . A hardware storage device having stored thereon computer executable instructions that are structured to be executable by a processor of a computing device to thereby cause the computing device to perform actions that:
track lines of command line interface (CLI) code entered into a CLI, wherein a line of CLI code comprises a command name followed by zero or one or more parameter strings;
cause a first deep learning model to generate a model-generated command name to follow a set of previous lines of CLI code entered into the CLI given the set of previous lines of CLI code;
output the model-generated command name in a user interface;
upon user input selecting the model-generated command name, cause a second deep learning model to generate a model-generated parameter string to follow the model-generated command name given the model-generated command name and the set of previous lines of CLI code entered into the CLI;
output the predicted command name and predicted parameter string as a syntactically-correct line of CLI code; and
upon user input selecting the syntactically-correct line of CLI code, input the syntactically-correct line of CLI code into the CLI.
9 . The hardware storage device of claim 8 , wherein the first deep learning model is generated from a pre-trained encoder model trained on an unsupervised dataset of source code snippets from a plurality of programming languages using a span masking objective and fine-tuned as a classifier using a supervised dataset of CLI code snippets to predict the command name.
10 . The hardware storage device of claim 8 , wherein the second deep learning model is generated from a pre-trained decoder model trained on an unsupervised dataset of source code snippets from a plurality of programming language using a conditional probability objective and fine-tuned on a supervised dataset of CLI code snippets.
11 . The hardware storage device of claim 8 , wherein the first deep learning model is an encoder-only neural transformer model with attention.
12 . The hardware storage device of claim 8 , wherein the second deep learning model is a decoder-only neural transformer model with attention.
13 . A computer-implemented method, comprising:
receiving last n lines of CLI code entered into the CLI;
causing a first deep learning model to generate k command names of CLI commands having a highest likelihood to follow the last n lines entered into the CLI as a syntactically-correct line of CLI code, wherein the deep learning model is given the last n lines of CLI code entered into the CLI;
outputting the k command names of the CLI commands to the CLI;
receiving a user selection of one of the k commands names of the CLI commands;
causing a second deep learning model to generate a parameter string to follow the user-selected one of the k command names given the user-selected one of the k command names;
presenting the user-selected command name and the parameter string as the syntactically-correct line of CLI code; and
upon user selection of the syntactically-correct line of CLI code, inputting the syntactically-correct line of CLI code into the CLI.
14 . The computer-implemented method of claim 13 , wherein the first deep learning model is pre-trained to generate the k command names to follow the last n lines of CLI code on an unsupervised dataset of source code snippets from a plurality of programming languages using a span masking objective.
15 . The computer-implemented method of claim 14 , wherein the pre-trained first deep learning model is fine-tuned on a supervised dataset of CLI code snippets to learn to generate a target command name.
16 . The computer-implemented method of claim 13 , wherein the n lines of CLI code are based on a number of lines of CLI code used to train the first deep learning model and/or the second deep learning model.
17 . The computer-implemented method of claim 13 , wherein the first deep learning model is an encoder-only neural transformer model with attention.
18 . The computer-implemented method of claim 13 , wherein the second deep learning model is based on a pre-trained decoder model having been trained on an unsupervised training dataset of source code programs from different programming languages.
19 . The computer-implemented method of claim 18 , wherein the second deep learning model is fine-tuned on a supervised training dataset of CLI code snippets.
20 . The computer-implemented method of claim 13 , wherein the second deep learning model is a decoder-only neural transformer model with attention.