Content retrieval standardization using generative artificial intelligence (AI) models

1 . A computer-implemented method for performing content item retrieval based on one or more generative artificial intelligence (AI) models, comprising:

in response to receiving a user query, generating a query intent prompt with instructions to generate a search query intent based on the user query and user profile information outputted in a standardized object format;

generating one or more content item prompts with instructions to generate categories of content items outputted in the standardized object format;

based on receiving a query intent response for the query intent prompt from a generative AI model and receiving one or more content category responses for the one or more content item prompts from the generative AI model, determining a set of content matches between the user query and the content items;

determining a correlation score for each content match in the set of content matches using a distilled encoder neural network; and

selecting a content item from the content items based on correlation scores.

2 . The computer-implemented method of claim 1 , further comprising receiving the user query from a client device associated with a user that includes a search request for content.

3 . The computer-implemented method of claim 2 , further comprising obtaining the user profile information in connection with receiving the user query, wherein the user profile information is associated with a user identifier of the user and includes user attributes, preferences, and characteristics.

4 . The computer-implemented method of claim 1 , further comprising:

providing the query intent prompt to the generative AI model, wherein the instructions in the query intent prompt direct the generative AI model to generate natural language text representative of user intent and user preferences in the standardized object format, and wherein the query intent prompt includes an example of a user query output generated in the standardized object format; and

receiving the query intent response from the generative AI model.

5 . The computer-implemented method of claim 1 , wherein a content item prompt includes a link to a given content item, content associated with the given content item, and metadata associated with the given content item.

6 . The computer-implemented method of claim 1 , further comprising:

identifying a set of content items that correspond to webpages; and

providing links to and content from the webpages within the one or more content item prompts.

7 . The computer-implemented method of claim 1 , further comprising:

providing the one or more content item prompts to the generative AI model, wherein the instructions in the one or more content item prompts direct the generative AI model to generate natural language text representative of content item category and content item attributes in the standardized object format, and wherein a content item prompt of the one or more content item prompts includes an example of a content item output generated in the standardized object format; and

receiving the one or more content category responses from the generative AI model, wherein the generative AI model is a same model used to generate user query outputs.

8 . The computer-implemented method of claim 1 , wherein:

the standardized object format is a JSON object file with properties and values;

a first JSON object file generated for the user query includes a first property with a first value; and

a second JSON object file generated for one or more content items of the content items includes a second property with the first value, the first property being different from the second property.

9 . The computer-implemented method of claim 1 , wherein determining the set of content matches includes comparing a first value of a first property in the query intent response with a second value of a second property in the one or more content category responses, wherein the first value matches and the second value.

10 . The computer-implemented method of claim 1 , wherein the generative AI model is a small action generative AI model trained to standardize input tokens and generate output tokens in the standardized object format.

11 . The computer-implemented method of claim 1 , wherein the distilled encoder neural network determines correlation scores for content matches between user query responses in the standardized object format and content category responses in the standardized object format.

12 . The computer-implemented method of claim 11 , further comprising utilizing the generative AI model to train the distilled encoder neural network to generate the correlation scores.

13 . The computer-implemented method of claim 1 , further comprising training the distilled encoder neural network by:

providing content matching pairs to the generative AI model with instructions to generate correlation scores for each content matching pair;

generating a training dataset with the content matching pairs and corresponding correlation scores; and

training the distilled encoder neural network in a supervised manner using the training dataset.

14 . The computer-implemented method of claim 1 , further comprising:

determining the content item by ranking the content items based on correlation scores; and

selecting the content item based on the content item having a highest correlation score.

15 . The computer-implemented method of claim 1 , further comprising retrieving the content item from the content items to provide to a client device associated with a user in response to the user query.

16 . A system comprising:

a processing system having a processor; and

a computer memory including instructions that, when executed by the processing system, cause the system to carry out operations comprising:

in response to receiving a user query, generating a query intent prompt with instructions to generate a search query intent based on the user query and user profile information outputted in a standardized object format;

generating one or more content item prompts with instructions to generate categories of content items outputted in the standardized object format;

based on receiving a query intent response for the query intent prompt from a generative AI model and receiving one or more content category responses for the one or more content item prompts from the generative AI model, determining a set of content matches between the user query and the content items;

determining a correlation score for each content match in the set of content matches using a distilled encoder neural network; and

selecting a content item from the content items based on correlation scores.

17 . The system of claim 16 , further comprising:

generating one or more content item prompts with instructions to generate categories of content items outputted in the standardized object format;

providing the one or more content item prompts to the generative AI model; and

receiving the one or more content category responses in the standardized object format from the generative AI model.

18 . A non-transitory computer-readable storage medium for performing content item retrieval based on one or more generative artificial intelligence (AI) models, comprising instructions that, when executed by a processor, cause a computing device to carry out operations comprising:

in response to receiving a user query, generating a query intent prompt with instructions to generate a search query intent based on the user query and user profile information outputted in a standardized object format;

generating one or more content item prompts with instructions to generate categories of content items outputted in the standardized object format;

based on receiving a query intent response for the query intent prompt from a generative AI model and receiving one or more content category responses for the one or more content item prompts from the generative AI model, determining a set of content matches between the user query and the content items;

determining a correlation score for each content match in the set of content matches using a distilled encoder neural network; and

selecting a content item from the content items based on correlation scores.

19 . The non-transitory computer-readable storage medium of claim 18 , wherein determining the set of content matches includes comparing a first value of a first property in the query intent response with a second value of a second property in the one or more content category responses, wherein the first value matches the second value.

20 . The non-transitory computer-readable storage medium of claim 19 , wherein the first property in the query intent response is different from the second property in the one or more content category responses.