Logical language model prompts for fraud detection
In some embodiments, a security chatbot comprises a generative language module (GLM) and a prompt manager configured to dynamically update the GLM input prompt according to the output of the GLM. The input prompt instructs the GLM to carry out a task (e.g., determine whether a target message received by a user is indicative of online fraud) and in response, to output a specific flag token. In response to detecting the flag token within the output of the GLM, the prompt manager may selectively execute a code snippet identified according to the respective flag token. Executing the code snippet effectively updates the prompt, which is then fed back to the GLM for the next inference step. In contrast to conventional LLM prompting, updates to the prompt herein comprise modifications (e.g., insertion of other flag tokens, etc.) other than merely appending an inferred token to the previous prompt. Furthermore, the prompt modifications are performed inline, i.e., without submitting an entirely new prompt to the GLM.
1. A computer system comprising at least one hardware processor configured to analyze a target text formulated in a natural language, wherein analyzing the target text comprises executing a prompt manager communicatively coupled to a generative language module (GLM), wherein:
the GLM is configured to:
receive from the prompt manager a language model (LM) prompt formulated in the natural language, and
in response, output to the prompt manager a predicted token comprising a continuation of the received LM prompt; and
the prompt manager is configured to:
receive a logical prompt, the logical prompt including the target text and a code snippet comprising computer code for updating the LM prompt,
formulate the LM prompt according to the logical prompt, the LM prompt instructing the GLM to perform a text analysis task according to the target text and in response, to output a flag token,
in response to receiving the predicted token, initiate an execution of the code snippet to produce an updated LM prompt,
transmit the updated LM prompt to the GLM, and
formulate a reply message comprising a result of analyzing the target text, the reply message formulated according to an output of the GLM in response to the updated LM prompt.
2. The computer system of claim 1 , wherein the execution of the code snippet causes the prompt manager to:
determine whether the predicted token includes the flag token; and
in response, if the predicted token includes the flag token, insert an LM instruction into the LM prompt, the LM instruction instructing the GLM to perform another text analysis task according to the target text.
3. The computer system of claim 2 , wherein the LM instruction further instructs the GLM to output another flag token in response to performing the other text analysis task.
4. The computer system of claim 1 , wherein:
the prompt manager is configured to formulate the LM prompt to further include a placeholder token; and
the execution of the code snippet causes the prompt manager to insert a set of supplemental tokens into the LM prompt at a position of the placeholder token.
5. The computer system of claim 1 , wherein the execution of the code snippet causes the prompt manager to perform an action selected from a group consisting of inserting a plurality of supplemental tokens into the LM prompt and deleting a set of tokens from the LM prompt.
6. The computer system of claim 1 , wherein the execution of the code snippet causes the prompt manager to delete the flag token from the LM prompt.
7. The computer system of claim 1 , wherein:
the logical prompt comprises a plurality of code snippets for updating the LM prompt, the plurality of code snippets including the code snippet;
the prompt manager is configured to associate each code snippet of the plurality of code snippets with a distinct flag token of a plurality of flag tokens; and
producing the updated LM prompt comprises executing a selected code snippet of the plurality of code snippets according to whether the predicted token includes a selected flag token of the plurality of flag tokens.
8. The computer system of claim 1 , wherein the flag token comprises an attribute-value pair.
9. The computer system of claim 1 , wherein the text analysis task comprises an item selected from a group consisting of determining whether the target text is indicative of online fraud, determining whether the target text comprises fake news, and determining whether the target text is indicative of hate speech.
10. A computer-implemented method of analyzing a target text formulated in a natural language, the method comprising employing at least one hardware processor of a computer system to execute a prompt manager communicatively coupled to a generative language module (GLM), wherein:
the GLM is configured to:
receive from the prompt manager a language model (LM) prompt formulated in the natural language, and
in response, output to the prompt manager a predicted token comprising a continuation of the received LM prompt; and
executing the prompt manager comprises:
receiving a logical prompt, the logical prompt including the target text and a code snippet comprising computer code for updating the LM prompt,
formulating the LM prompt according to the logical prompt, the LM prompt instructing the GLM to perform a text analysis task according to the target text and in response, to output a flag token,
in response to receiving the predicted token, initiating an execution of the code snippet to produce an updated LM prompt,
transmitting the updated LM prompt to the GLM, and
formulating a reply message comprising a result of analyzing the target text, the reply message formulated according to an output of the GLM in response to the updated LM prompt.
11. The method of claim 10 , wherein the execution of the code snippet causes the prompt manager to:
determine whether the predicted token includes the flag token; and
in response, if the predicted token includes the flag token, insert an LM instruction into the LM prompt, the LM instruction instructing the GLM to perform another text analysis task according to the target text.
12. The method of claim 11 , wherein the LM instruction further instructs the GLM to output another flag token in response to performing the other text analysis task.
13. The method of claim 10 , wherein:
the prompt manager is configured to formulate the LM prompt to further include a placeholder token; and
the execution of the code snippet causes the prompt manager to insert a set of supplemental tokens into the LM prompt at a position of the placeholder token.
14. The method of claim 10 , wherein the execution of the code snippet causes the prompt manager to perform an action selected from a group consisting of inserting a plurality of supplemental tokens into the LM prompt and deleting a set of tokens from the LM prompt.
15. The method of claim 10 , wherein the execution of the code snippet causes the prompt manager to delete the flag token from the LM prompt.
16. The method of claim 10 , wherein:
the logical prompt comprises a plurality of code snippets including the code snippet;
the prompt manager is configured to associate each code snippet of the plurality of code snippets with a distinct flag token of a plurality of flag tokens; and
producing the updated LM prompt comprises executing a selected code snippet of the plurality of code snippets according to whether the predicted token includes a selected flag token of the plurality of flag tokens.
17. The method of claim 10 , wherein the flag token comprises an attribute-value pair.
18. The method of claim 10 , wherein the text analysis task comprises an item selected from a group consisting of determining whether the target text is indicative of online fraud, determining whether the target text comprises fake news, and determining whether the target text is indicative of hate speech.
19. A non-transitory computer-readable medium storing instructions which, when executed by at least one hardware processor of a computer system, cause the computer system to analyze a target text formulated in a natural language, wherein analyzing the target text comprises executing a prompt manager communicatively coupled to a generative language module (GLM), wherein:
the GLM is configured to:
receive from the prompt manager a language model (LM) prompt formulated in a natural language, and
in response, output to the prompt manager a predicted token comprising a continuation of the received LM prompt; and
the prompt manager is configured to:
receive a logical prompt, the logical prompt including the target text and a code snippet comprising computer code for updating the LM prompt,
formulate the LM prompt according to the logical prompt, the LM prompt instructing the GLM to perform a text analysis task according to the target text and in response, to output a flag token,
in response to receiving the predicted token, initiate an execution of the code snippet to produce an updated LM prompt,
transmit the updated LM prompt to the GLM, and
formulate a reply message comprising a result of analyzing the target text, the reply message formulated according to an output of the GLM in response to the updated LM prompt.