Generative artificial intelligence model streaming

1 . A system comprising:

a non-transitory data storage medium to store specific computer-executable instructions; and

a computer hardware processor in communication with the non-transitory data storage medium, wherein the computer hardware processor is to execute the specific computer-executable instructions to at least:

receive an input query from a user computing device;

determine a question type for the input query;

determine a prompt based at least in part on the question type and the input query;

provide the prompt to a trained generative model;

receive, in a stream from the trained generative model, first content comprising a plurality of tokens, wherein a first token of the plurality of tokens is an intermediate token;

determine first markup content from the first content, wherein to determine the first markup content, the computer hardware processor is to execute the computer-executable instructions to at least:

replace, in the plurality of tokens, the intermediate token with markup content;

transmit the first markup content to the user computing device at a first time;

receive, in the stream from the trained generative model, second content; and

transmit at least some of the second content to the user computing device at a second time.

2 . The system of claim 1 , wherein the computer hardware processor executes further computer-executable instructions to at least:

determine an item associated with the input query, wherein to determine the prompt, the computer hardware processor is to execute the further computer-executable instructions to at least:

insert the item into a prompt template.

3 . The system of claim 2 , wherein to determine the first markup content, the computer hardware processor is to execute additional computer-executable instructions to at least:

insert, at a placeholder location in the plurality of tokens, a title associated with the item.

4 . The system of claim 2 , wherein to determine the first markup content, the computer hardware processor is to execute additional computer-executable instructions to at least:

insert, at a placeholder location in the plurality of tokens, a resource locator for an image associated with the item.

5 . The system of claim 1 , wherein the computer hardware processor executes further computer-executable instructions to at least:

determine a related question associated with the input query, wherein to determine the prompt, the computer hardware processor is to execute the further computer-executable instructions to at least:

insert the related question into a prompt template.

6 . The system of claim 5 , wherein to determine the first markup content, the computer hardware processor is to execute additional computer-executable instructions to at least:

insert, at a placeholder location in the plurality of tokens, the related question.

7 . A computer-implemented method comprising:

receiving an input query from a user computing device;

determining a question type for the input query;

determining first input based at least in part on the question type and the input query;

providing the first input to a trained generative model;

receiving, in a stream from the trained generative model, first content comprising a plurality of tokens, wherein a first token of the plurality of tokens is an intermediate token;

determining, from the first content, first output content in a first data format different from a data format of the first content, wherein determining the first output content further comprises:

replacing, in the plurality of tokens, the intermediate token with content in the first data format;

transmitting the first output content to the user computing device at a first time;

receiving, in the stream from the trained generative model, second content; and

transmitting at least some of the second content to the user computing device at a second time.

8 . The computer-implemented method of claim 7 , further comprising:

retraining a pre-trained language model with machine learning and a training data set comprising a prompt and response pair, wherein the prompt and response pair comprises the intermediate token, wherein retraining the pre-trained language model further comprises:

outputting the trained generative model.

9 . The computer-implemented method of claim 7 , wherein determining the first input further comprises:

selecting a prompt template associated with the question type; and

inserting, into the prompt template, at least some of the input query.

10 . The computer-implemented method of claim 7 , further comprising:

determining an item associated with the input query, wherein determining the first output content further comprises:

inserting, into intermediate content, a placeholder token; and

replacing the placeholder token with metadata associated with the item.

11 . The computer-implemented method of claim 10 , wherein the metadata comprises a resource locator for the item.

12 . The computer-implemented method of claim 7 , further comprising:

determining a user profile associated with the input query; and

determining a communication history associated with the user profile, wherein determining the first input further comprises:

determining second input based at least in part on the communication history.

13 . The computer-implemented method of claim 7 , further comprising:

determining a user profile associated with the input query; and

determining a navigation history associated with the user profile, wherein determining the first input further comprises:

determining second input based at least in part on the navigation history.

14 . A system comprising:

a non-transitory data storage medium to store specific computer-executable instructions; and

a computer hardware processor in communication with the non-transitory data storage medium, wherein the computer hardware processor is to execute the specific computer-executable instructions to at least:

receive an input query from a user computing device;

determine a question type for the input query;

determine first input based at least in part on the question type and the input query;

provide the first input to a trained generative model;

receive, in a stream from the trained generative model, first content comprising a plurality of tokens, wherein a first token of the plurality of tokens is an intermediate token;

determine, from the first content, first output content in a first data format different from a data format of the first content, wherein to determine the first output content, the computer hardware processor is to execute the computer-executable instructions to at least:

replace, in the plurality of tokens, the intermediate token with content in the first data format;

transmit the first output content to the user computing device at a first time;

receive, in the stream from the trained generative model, second content; and

transmit at least some of the second content to the user computing device at a second time.

15 . The system of claim 14 , wherein the first data format comprises a markup format or an object notation format.

16 . The system of claim 14 , wherein the computer hardware processor executes further computer-executable instructions to at least:

retrain a pre-trained language model with machine learning and a training data set comprising a prompt and response pair, wherein the prompt and response pair comprises the intermediate token, wherein to retrain the pre-trained language model, the computer hardware processor is to execute the further computer-executable instructions to at least:

output the trained generative model.

17 . The system of claim 14 , wherein the computer hardware processor executes further computer-executable instructions to at least:

determine an item associated with the input query, wherein to determine the first input, the computer hardware processor is to execute the further computer-executable instructions to at least:

determine second input based at least in part on the item.

18 . The system of claim 17 , wherein to determine the first output content, the computer hardware processor is to execute additional computer-executable instructions to at least:

insert, at a placeholder location in the plurality of tokens, a title associated with the item.

19 . The system of claim 17 , wherein to determine the first output content, the computer hardware processor is to execute further computer-executable instructions to at least:

insert, into intermediate content, a placeholder token; and

replace the placeholder token with metadata associated with the item.

20 . The system of claim 14 , wherein the computer hardware processor executes further computer-executable instructions to at least:

determine a related question associated with the input query, wherein to generate the first output content, the computer hardware processor is to execute the further computer-executable instructions to at least:

insert, at a placeholder location in the plurality of tokens, the related question.