Applying transformations on streaming output
Techniques for streaming generative machine learned model (or LLM) output to a virtual space are described herein. A system may receive a request to perform an action. The system may leverage LLMs to assist in performing aspects of the requested action. The system can generate input data to input into the LLM. When generating the input data, the system can identify sensitive data associated with the request. The system can modify the input data to mask and/or anonymize the sensitive data. The system can input the input data into an LLM trained to output a subset (e.g., less than all) of the response at a time. The system can add the output subset to a buffer and upon identifying the masked data in the buffer, the system can demask the sensitive data and output the sensitive data to the user profile in a streaming manner.
1 . A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause the one or more processors to perform operations comprising:
receiving, from a user profile of a user, a request for a generative machine learned model to perform an action;
identifying, prior to inputting data associated with the request into the generative machine learned model, sensitive data in the data;
generating, in response to identifying the sensitive data, modified data that includes a mask to anonymize the sensitive data;
inputting the modified data into the generative machine learned model that is configured to output a response;
receiving, from the generative machine learned model, a subset of the response;
causing the subset of the response to be added to a buffer;
determining, based at least in part on determining that the buffer is full, that the buffer includes the mask;
generating, in response to determining that the buffer includes the mask, a modified subset of the response that includes the sensitive data; and
causing, in response to generating the modified subset of the response, the modified subset to be output to the user profile.
2 . The system of claim 1 , wherein generating the modified subset of the response is based at least in part on:
determining a buffer size associated with the buffer; and
determining, based at least in part on the buffer size, that the buffer is full, wherein generating the modified subset is based at least in part on the buffer being full.
3 . The system of claim 2 , wherein determining the buffer size is based at least in part on:
receiving one or more mappings that include one or more keys with associated values;
identifying, based at least in part on the one or more mappings, a size of a key of the one or more keys that includes a largest amount of characters; and
generating the buffer size that corresponds to the size.
4 . The system of claim 1 , wherein generating the modified subset of the response is based at least in part on:
receiving one or more mappings that include one or more keys with associated values;
comparing the one or more keys to contents associated with the buffer;
determining, based at least in part on comparing the one or more keys to the contents of the buffer, that a key of the one or more keys match the contents in the buffer; and
generating, based at least in part on the key matching the contents in the buffer, the modified subset of the response.
5 . The system of claim 1 , the operations further comprising:
receiving one or more mappings that include one or more keys with associated values;
comparing the one or more keys to contents associated with the buffer;
determining, based at least in part on comparing the one or more keys to the contents of the buffer, that the contents of the buffer are different than the one or more keys; and
causing, in response to determining that the contents of the buffer are different than the one or more keys, a token of the buffer to be output to the user profile.
6 . The system of claim 1 , wherein the sensitive data comprises at least one of:
a user name,
an organization name,
a phone number,
an address,
government issued data, or
financial data associated with the user.
7 . One or more non transitory computer readable media storing instructions executable by one or more processors, wherein the instructions, when executed, cause the one or more processors to perform operations comprising:
receiving, from a user profile of a user, a request for a generative machine learned model to perform an action;
identifying, prior to inputting data associated with the request into the generative machine learned model, sensitive data in the data;
generating, in response to identifying the sensitive data, modified data that includes a mask to anonymize the sensitive data;
inputting the modified data into the generative machine learned model that is configured to output a response;
receiving, from the generative machine learned model, a subset of the response;
determining, based at least in part on the subset of the response and determining that a buffer is full, that the buffer includes the mask;
generating, in response to determining that the buffer includes the mask, a modified subset of the response that includes the sensitive data; and
causing, in response to generating the modified subset of the response, the modified subset to be output to the user profile.
8 . The one or more non transitory computer readable media of claim 7 , wherein generating the modified subset is based at least in part on:
causing the subset of the response to be added to the buffer; and
determining that the buffer includes the mask, wherein generating the modified subset is based at least in part on determining that the buffer includes the mask.
9 . The one or more non transitory computer readable media of claim 8 , wherein generating the modified subset of the response is based at least in part on:
determining a buffer size associated with the buffer; and
determining, based at least in part on the buffer size, that the buffer is full, wherein generating the modified subset is based at least in part on the buffer being full.
10 . The one or more non transitory computer readable media of claim 9 , wherein determining the buffer size is based at least in part on:
receiving one or more mappings that include one or more keys with associated values;
identifying, based at least in part on the one or more mappings, a size of a key of the one or more keys that includes a largest amount of characters; and
generating the buffer size that corresponds to the size.
11 . The one or more non transitory computer readable media of claim 8 , wherein generating the modified subset of the response is based at least in part on:
receiving one or more mappings that include one or more keys with associated values;
comparing the one or more keys to contents associated with the buffer;
determining, based at least in part on comparing the one or more keys to the contents of the buffer, that a key of the one or more keys match the contents in the buffer; and
generating, based at least in part on the key matching the contents in the buffer, the modified subset of the response.
12 . The one or more non transitory computer readable media of claim 8 , the operations further comprising:
receiving one or more mappings that include one or more keys with associated values;
comparing the one or more keys to contents associated with the buffer;
determining, based at least in part on comparing the one or more keys to the contents of the buffer, that the contents of the buffer are different than the one or more keys; and
causing, in response to determining that the contents of the buffer are different than the one or more keys, a token of the buffer to be output to the user profile.
13 . The one or more non transitory computer readable media of claim 7 , wherein the sensitive data comprises at least one of:
a user name,
an organization name,
a phone number,
an address,
government issued data, or
financial data associated with the user.
14 . A method comprising:
receiving, from a user profile of a user, a request for a generative machine learned model to perform an action;
identifying, prior to inputting data associated with the request into the generative machine learned model, sensitive data in the data;
generating, in response to identifying the sensitive data, modified data that includes a mask to anonymize the sensitive data;
inputting the modified data into the generative machine learned model that is configured to output a response;
receiving, from the generative machine learned model, a subset of the response;
determining, based at least in part on the subset of the response and determining that a buffer is full, that the buffer includes the mask;
generating, in response to determining that the buffer includes the mask, a modified subset of the response that includes the sensitive data; and
causing, in response to generating the modified subset of the response, the modified subset to be output to the user profile.
15 . The method of claim 14 , wherein generating the modified subset is based at least in part on:
causing the subset of the response to be added to the buffer; and
determining that the buffer includes the mask, wherein generating the modified subset is based at least in part on determining that the buffer includes the mask.
16 . The method of claim 15 , wherein generating the modified subset of the response is based at least in part on:
determining a buffer size associated with the buffer; and
determining, based at least in part on the buffer size, that the buffer is full, wherein generating the modified subset is based at least in part on the buffer being full.
17 . The method of claim 16 , wherein determining the buffer size is based at least in part on:
receiving one or more mappings that include one or more keys with associated values;
identifying, based at least in part on the one or more mappings, a size of a key of the one or more keys that includes a largest amount of characters; and
generating the buffer size that corresponds to the size.
18 . The method of claim 15 , wherein generating the modified subset of the response is based at least in part on:
receiving one or more mappings that include one or more keys with associated values;
comparing the one or more keys to contents associated with the buffer;
determining, based at least in part on comparing the one or more keys to the contents of the buffer, that a key of the one or more keys match the contents in the buffer; and
generating, based at least in part on the key matching the contents in the buffer, the modified subset of the response.
19 . The method of claim 15 , further comprising:
receiving one or more mappings that include one or more keys with associated values;
comparing the one or more keys to contents associated with the buffer;
determining, based at least in part on comparing the one or more keys to the contents of the buffer, that the contents of the buffer are different than the one or more keys; and
causing, in response to determining that the contents of the buffer are different than the one or more keys, a token of the buffer to be output to the user profile.
20 . The method of claim 14 , wherein the sensitive data comprises at least one of:
a user name,
an organization name,
a phone number,
an address,
government issued data, or
financial data associated with the user.