Customized prompt generation service for software engineering tasks

1 . A system for automatic generation of a prompt with customized data, said system comprising:

one or more processors; and

one or more hardware storage devices that store instructions that are executable by the one or more processors to cause the system to:

provide a plurality of services, wherein each service is configured to generate a prompt to a large language model for a select software engineering task to be performed by the large language model, wherein each service of the plurality of services is associated with a prompt template, and wherein the prompt template identifies content to be included in the prompt;

receive a request to perform a first software engineering task using the large language model, wherein the request is associated with a client, and wherein the first software engineering task includes identifying a vulnerability in source code;

obtain a prompt template associated with the first software engineering task;

obtain a retrieval-augmented example of the first software engineering task from custom data of the client;

generate an initial prompt for the large language model to perform the first software engineering task based on the prompt template, wherein the initial prompt includes the content specified in the prompt template and the retrieval-augmented example of the first software engineering task from the custom data of the client;

transmit the initial prompt to the large language model for the large language model to perform the first software engineering task given the content of the initial prompt;

in response to identifying the vulnerability in the source code, determine a vulnerability type for the vulnerability;

generate an embedding for the vulnerability type;

use the embedding for the vulnerability type to find a top-k closest matching set of embeddings within a retrieval-augmented index, wherein the top-k closest matching set of embeddings identified within the retrieval-augmented index are ones that point to repair code that is usable to repair vulnerabilities classified as having the same vulnerability type, such that said vulnerability is potentially repairable using one or more of the top-k closest matching set of embeddings; and

compile the repair code and determine whether a unit test is attached in the request, wherein, when the repair code does not compile successfully or when the unit test is not executed successfully, a given service continues conversation with the large language model to fix the repair code.

2 . The system of claim 1 , wherein the instructions are further executable to cause the system to:

obtain a response from the large language model; and

check accuracy of the response.

3 . The system of claim 2 , wherein the instructions are further executable to cause the system to:

generate a second prompt to the large language model when the response to the first prompt contains an erroneous response, wherein the second prompt includes the first prompt, the response to the first prompt, and instructions to correct the response to the first prompt.

4 . The system of claim 1 , wherein the plurality of services comprises a code review prompt service, a code search prompt service, a vulnerability detection prompt service, and/or a unit test generation prompt service.

5 . The system of claim 1 , wherein the instructions are further executable to cause the system to:

store the custom data in a retrieval-augmented data store, wherein an item in the retrieval-augmented data store is associated with a second embedding used to retrieve the item.

6 . The system of claim 5 , wherein the retrieval-augmented data store includes custom data of the client comprising code segments, unit tests, code reviews, and code diff hunks.

7 . The system of claim 1 , wherein the large language model is a neural transformer model with attention.

8 . The system of claim 1 , wherein the large language model is a conversational generative pre-trained neural transformer model with attention.

9 . A computer-implemented method for automatic generation of a prompt with customized data, said method comprising:

hosting a plurality of web services, wherein a web service is associated with generating a prompt to a large language model to perform a specified software engineering task, wherein a web service is associated with a prompt template specifying content to be included in the prompt;

hosting customized data of a client, wherein the customized data comprises source code artifacts from a codebase of the client;

receiving a request to perform an action on a first source code artifact of the client;

identifying a first software engineering task to perform the requested action, wherein the first software engineering task includes identifying a vulnerability in the first source code artifact;

obtaining, from the customized data of the client, the first source code artifact from the codebase of the client illustrating the first software engineering task;

generating a first prompt to the large language model to perform the first software engineering task based on the content of the prompt template associated with the first software engineering task, wherein the first prompt includes the first source code artifact of the client;

transmitting the first prompt to the large language model;

receiving a response from the large language model including an output of the first software engineering task;

in response to identifying the vulnerability in the first source code artifact, determining a vulnerability type for the vulnerability;

generating an embedding for the vulnerability type;

using the embedding for the vulnerability type to find a top-k closest matching set of embeddings within a retrieval-augmented index, wherein the top-k closest matching set of embeddings identified within the retrieval-augmented index are ones that point to repair code that is usable to repair vulnerabilities classified as having the same vulnerability type, such that said vulnerability is potentially repairable using one or more of the top-k closest matching set of embeddings; and

compiling the repair code and determining whether a unit test is attached in the request, wherein, when the repair code does not compile successfully or when the unit test is not executed successfully, a given service continues conversation with the large language model to fix the repair code.

10 . The computer-implemented method of claim 9 , further comprising:

processing the output of the first software engineering task for accuracy; and

upon determining the output is accurate, outputting the output of the first software engineering task.

11 . The computer-implemented method of claim 9 , further comprising:

processing the output of the first software engineering task for compliance with the request;

upon determining the output is non-compliant, generating a second prompt to correct the response, wherein the second prompt includes the first prompt and the response to the first prompt.

12 . The computer-implemented method of claim 9 , further comprising:

accessing the first source code artifact of the client utilizing a second embedding associated with the request having a threshold level of similarity relative to a third embedding of the first source code artifact.

13 . The computer-implemented method of claim 9 , wherein the source code artifacts comprise code changes made to the codebase, code segments of the codebase, unit tests, and/or code reviews.

14 . The computer-implemented method of claim 9 , wherein the plurality of web services comprises a code review prompt service, a code search prompt service, a vulnerability detection prompt service, and/or a unit test generation prompt service.

15 . The computer-implemented method of claim 9 , wherein the large language model is a conversational generative pre-trained neural transformer model with attention.

16 . A hardware storage device storing computer-executable instructions that are structured to be executed by a processor of a web service to cause the web service to:

store a plurality of prompt templates, each prompt template associated with a software engineering task and specifies content to be included in a prompt for a large language model to perform the associated software engineering task;

store a customized data of a client;

receive a request to perform a first software engineering task that includes identifying a vulnerability in source code;

retrieve the customized data of the client associated with the first software engineering task;

generate a first prompt containing the content specified in the prompt template for the first software engineering task and the customized data of the client associated with the software engineering task;

obtain an output of the software engineering task from the large language model given the first prompt;

in response to identifying the vulnerability in the source code, determine a vulnerability type for the vulnerability;

generate an embedding for the vulnerability type;

use the embedding for the vulnerability type to find a top-k closest matching set of embeddings within a retrieval-augmented index, wherein the top-k closest matching set of embeddings identified within the retrieval-augmented index are ones that point to repair code that is usable to repair vulnerabilities classified as having the same vulnerability type, such that said vulnerability is potentially repairable using one or more of the top-k closest matching set of embeddings; and

compile the repair code and determine whether a unit test is attached in the request, wherein, when the repair code does not compile successfully or when the unit test is not executed successfully, a given service continues conversation with the large language model to fix the repair code.

17 . The hardware storage device of claim 16 , wherein the instructions are further executable to cause the web service to:

generate a second embedding for the customized data to serve as an index to access the customized data.

18 . The hardware storage device of claim 16 , wherein the software engineering task comprises code review generation, unit test generation, vulnerability detection, or code search.

19 . The hardware storage device of claim 16 , wherein the customized data includes code changes, code reviews, unit tests, code segments, and/or repair code from a codebase of the client.

20 . The hardware storage device of claim 16 , wherein the large language model is a conversational generative pre-trained neural transformer model with attention.