Dynamic Language Model Prompts for Fraud Detection
In some embodiments, a security chatbot comprises a generative language module (GLM) and a prompt manager configured to dynamically update the GLM input prompt according to the output of the GLM. The input prompt instructs the GLM to carry out a task (e.g., determine whether a target message received by a user is indicative of online fraud) and in response, to output a specific flag token. In response to detecting the flag token within the output of the GLM, the prompt manager may selectively execute a code snippet identified according to the respective flag token. Executing the code snippet effectively updates the prompt, which is then fed back to the GLM for the next inference step. In contrast to conventional LLM prompting, updates to the prompt herein comprise modifications (e.g., insertion of other flag tokens, etc.) other than merely appending an inferred token to the previous prompt. Furthermore, the prompt modifications are performed inline, i.e., without submitting an entirely new prompt to the GLM.
1 . A computer system comprising at least one hardware processor configured to execute a chatbot configured to output a verdict formulated in a natural language and indicating whether a received target message is indicative of fraud, the chatbot comprising a prompt manager communicatively coupled to a generative language module (GLM), wherein:
the GLM is configured to:
receive from the prompt manager a prompt formulated in the natural language, and
in response, output to the prompt manager a predicted token comprising a likely continuation of the received prompt; and
the prompt manager is configured to:
formulate the prompt to instruct the GLM to perform a fraud detection task according to the target message,
determine whether the predicted token comprises a pre-determined flag token,
in response, if the predicted token comprises the flag token, initiate an execution of a code snippet, wherein executing the code snippet causes an update of the prompt,
transmit the updated prompt to the GLM, and
determine the verdict according to an output produced by the GLM in response to the updated prompt.
2 . The computer system of claim 1 , wherein the prompt manager is configured to formulate the prompt to instruct the GLM to determine whether the target message is indicative of fraud.
3 . The computer system of claim 2 , wherein the prompt manager is configured to formulate the prompt to further instruct the GLM to output the flag token in response to a determination by the GLM that the target message is indicative of fraud.
4 . The computer system of claim 3 , wherein the prompt manager is configured to formulate the prompt to further instruct the GLM to output another pre-determined flag token in response to a determination by the GLM that the target message is not indicative of fraud.
5 . The computer system of claim 1 , wherein:
the prompt manager is configured to formulate the prompt to further include a placeholder token; and
the update of the prompt comprises inserting a set of supplemental tokens into the prompt at a position of the placeholder token.
6 . The computer system of claim 1 , wherein the update of the prompt comprises an action selected from a group consisting of inserting a plurality of supplemental tokens into the prompt and deleting a set of tokens from the prompt.
7 . The computer system of claim 1 , wherein the update of the prompt comprises inserting a sequence of supplemental tokens into the prompt, the sequence of supplemental tokens instructing the GLM to insert another pre-determined flag token into the prompt.
8 . The computer system of claim 1 , wherein:
the prompt manager is configured to formulate the prompt to instruct the GLM to output the flag token according to a result of the fraud detection task; and
the update of the prompt instructs the GLM to perform another fraud detection task.
9 . The computer system of claim 1 , wherein the prompt manager is configured to select the code snippet from a plurality of pre-determined code snippets according to the flag token.
10 . The computer system of claim 1 , wherein the flag token comprises an attribute-value pair.
11 . A computer-implemented method comprising employing at least one hardware processor of a computer system to execute a chatbot configured to output a verdict formulated in a natural language and indicating whether a received target message is indicative of fraud, the chatbot comprising a prompt manager communicatively coupled to a generative language module (GLM), wherein:
the GLM is configured to:
receive from the prompt manager a prompt formulated in the natural language, and
in response, output to the prompt manager a predicted token comprising a likely continuation of the received prompt; and
executing the prompt manager comprises:
formulating the prompt to instruct the GLM to perform a fraud detection task according to the target message,
determining whether the predicted token comprises a pre-determined flag token,
in response, if the predicted token comprises the flag token, initiating an execution of a code snippet, wherein executing the code snippet causes an update of the prompt,
transmitting the updated prompt to the GLM, and
determining the verdict according to an output produced by the GLM in response to the updated prompt.
12 . The method of claim 11 , wherein executing the prompt manager comprises formulating the prompt to instruct the GLM to determine whether the target message is indicative of fraud.
13 . The method of claim 12 , wherein executing the prompt manager comprises formulating the prompt to further instruct the GLM to output the flag token in response to a determination by the GLM that the target message is indicative of fraud.
14 . The method of claim 13 , wherein executing the prompt manager comprises formulating the prompt to further instruct the GLM to output another pre-determined flag token in response to a determination by the GLM that the target message is not indicative of fraud.
15 . The method of claim 11 , wherein:
executing the prompt manager comprises formulating the prompt to further include a placeholder token; and
the update of the prompt comprises inserting a set of supplemental tokens into the prompt at a position of the placeholder token.
16 . The method of claim 11 , wherein the update of the prompt comprises an action selected from a group consisting of inserting a plurality of supplemental tokens into the prompt and deleting a set of tokens from the prompt.
17 . The method of claim 11 , wherein the update of the prompt comprises inserting a sequence of supplemental tokens into the prompt, the sequence of supplemental tokens instructing the GLM to insert another pre-determined flag token into the prompt.
18 . The method of claim 11 , wherein:
executing the prompt manager comprises formulating the prompt to instruct the GLM to output the flag token according to a result of the fraud detection task; and
the update of the prompt instructs the GLM to perform another fraud detection task.
19 . The method of claim 11 , wherein the prompt manager is configured to select the code snippet from a plurality of pre-determined code snippets according to the flag token.
20 . The method of claim 11 , wherein the flag token comprises an attribute-value pair.
21 . A non-transitory computer-readable medium storing instructions which, when executed by at least one hardware processor of a computer system, cause the computer system to form a chatbot configured to output a verdict formulated in a natural language and indicating whether a received target message is indicative of fraud, the chatbot comprising a prompt manager communicatively coupled to a generative language module (GLM), wherein:
the GLM is configured to:
receive from the prompt manager a prompt formulated in the natural language, and
in response, output to the prompt manager a predicted token comprising a likely continuation of the received prompt; and
the prompt manager is configured to:
formulate the prompt to instruct the GLM to perform a fraud detection task according to the target message,
determine whether the predicted token comprises a pre-determined flag token,
in response, if the predicted token comprises the flag token, initiate an execution of a code snippet, wherein executing the code snippet causes an update of the prompt,
transmit the updated prompt to the GLM, and
determine the verdict according to an output produced by the GLM in response to the updated prompt.