Granted Patent B1

US 12,306,842 · App. 18/943,304 · Granted May 20, 2025

Inventor: Andrew Donald Yates (San Francisco, CA)

Assignee: Promoted.ai, Inc.

G06F16/24578G06N20/00

Abstract

Claims (66)

1. A computer-implemented method performed by one or more processors, comprising the operations of:

receiving a natural language-based input associated with a client device of a user;

generating, by the one or more processors, a search criterion for the received natural language-based input;

generating, by the one or more processors, a relevancy-ranked output of content items, wherein generating the relevancy-ranked output comprises:

causing the generated search criterion to be processed, via a generative Artificial Intelligence (AI) search system, comprising one or more machine learning models;

generating by the generative AI search system, a relevancy-ranked output listing content items responsive to the generated search criterion, wherein the content items have a content identifier and a content description; and

causing a portion of the relevancy-ranked output listing to be rendered at the client device of the user;

generating, by the one or more processors, a search summary indicating the content identifiers, the content description and an associated relevancy ranking value of the content items associated with the generated relevancy-ranked output;

receiving, via a user interface, a modified relevancy ranking value for one or more of the content items indicated in the search summary, wherein a relevancy ranking value is based on a scale of multiple numeric relevancy ranking values, wherein one ranking value on the scale indicates that a content item is irrelevant to the generated search criterion, wherein another ranking value on the scale indicates that a content item is exactly relevant to the generated search criterion, and wherein the user interface displays multiple content items and the generated search criterion; and

retraining, by the one or more processors, the one or more machine learning models with the received modified relevancy ranking value, wherein the retrained one or more machine learning models generates a different relevancy-ranked output when the search criterion is applied to the generative AI search system.

2. The computer-implemented method of 1 , wherein causing the generated search criterion to be processed, via a search system, comprises the operations of

performing, by the one or more processors, a first stage scoring process, to generate a set of content items; and

performing, by the one or more processors, a second stage scoring process, by processing the set of content items by an inferencing machine learning model trained to determine item score values; and

generating, by the inferencing machine learning model, item score values for the set of content items.

3. The computer-implemented method of claim 2 , further comprising the operations of:

performing, by the one or more processors, a third stage scoring process that modifies an item score value to adjust a content item position placement in the relevancy-ranked output listing.

4. The computer-implemented method of claim 2 , further comprising the operations of:

performing, by the one or more processors, a blender process that generates annotations for one or more of the content items in the relevancy-ranked output listing, wherein the generated annotations comprise a type of content item for a respective content item in the relevancy-ranked output listing.

5. The computer-implemented method of claim 2 , wherein the first stage scoring process comprises:

generating, by the one or more processors, a search vector embedding of the received input;

performing a vector similarity matching of the search vector embedding as to a set of vector embeddings describing the content items; and

generating a listing of those content items where a similarity threshold match value is met or exceeded.

6. A system comprising one or more processors configured to perform the operations of:

receiving a natural language-based input associated with a client device of a user;

generating, by the one or more processors, a search criterion for the received natural language-based input;

generating, by the one or more processors, a relevancy-ranked output of content items, wherein generating the relevancy-ranked output comprises:

causing the generated search criterion to be processed, via a generative Artificial Intelligence (AI) search system, comprising one or more machine learning models;

generating by the generative AI search system, a relevancy-ranked output listing content items responsive to the generated search criterion, wherein the content items have a content identifier and a content description; and

causing a portion of the relevancy-ranked output listing to be rendered at the client device of the user;

generating, by the one or more processors, a search summary indicating the content identifiers, the content description and an associated relevancy ranking value of the content items associated with the generated relevancy-ranked output;

receiving, via a user interface, a modified relevancy ranking value for one or more of the content items indicated in the search summary, wherein a relevancy ranking value is based on a scale of multiple numeric relevancy ranking values, wherein

one ranking value on the scale indicates that a content item is irrelevant to the generated search criterion, wherein another ranking value on the scale indicates that a content item is exactly relevant to the generated search criterion, and wherein the user interface displays multiple content items and the generated search criterion; and

retraining, by the one or more processors, the one or more machine learning models with the received modified relevancy ranking value, wherein the retrained one or more machine learning models generates a different relevancy-ranked output when the search criterion is applied to the generative AI search system.

7. The system of claim 6 , wherein causing the generated search criterion to be processed, via a search system, comprises:

performing, by the one or more processors, a first stage scoring process, to generate a set of content items; and

performing, by the one or more processors, a second stage scoring process, by processing the set of content items by an inferencing machine learning model trained to determine item score values; and

generating, by the inferencing machine learning model, item score values for the set of content items.

8. The system of claim 7 , further comprising the operations of:

performing, by the one or more processors, a third stage scoring process that modifies an item score value to adjust a content item position placement in the relevancy-ranked output listing.

9. The system of claim 7 , further comprising the operations of:

performing, by the one or more processors, a blender process that generates annotations for one or more of the content items in the relevancy-ranked output listing, wherein the generated annotations comprise a type of content item for a respective content item in the relevancy-ranked output listing.

10. The system of claim 7 , wherein the first stage scoring process comprises:

generating, by the one or more processors, a search vector embedding of the received input;

performing a vector similarity matching of the search vector embedding as to a set of vector embeddings describing the content items; and

generating a listing of those content items where a similarity threshold match value is met or exceeded.

11. A non-transitory computer readable medium storing a software program comprising data and computer implementable instructions that when executed by at least one processor cause the at least one processor to perform operations receiving a natural language-based input associated with a client device of a user;

generating, by the one or more processors, a search criterion for the received natural language-based input;

generating, by the one or more processors, a relevancy-ranked output of content items, wherein generating the relevancy-ranked output comprises:

causing the generated search criterion to be processed, via a generative Artificial Intelligence (AI) search system, comprising one or more machine learning models;

generating by the generative AI search system, a relevancy-ranked output listing content items responsive to the generated search criterion, wherein the content items have a content identifier and a content description; and

causing a portion of the relevancy-ranked output listing to be rendered at the client device of the user; and

generating a search summary indicating the content identifiers, the content description and an associated relevancy ranking value of the content items associated with the generated relevancy-ranked output;

receiving, via a user interface, a modified relevancy ranking value for one or more of the content items indicated in the search summary, wherein a relevancy ranking value is based on a scale of multiple numeric relevancy ranking values, wherein one ranking value on the scale indicates that a content item is irrelevant to the generated search criterion, wherein another ranking value on the scale indicates that a content item is exactly relevant to the generated search criterion, and wherein the user interface displays multiple content items and the generated search criterion; and

retraining, by the one or more processors, the one or more machine learning models with the received modified relevancy ranking value, wherein the retrained one or more machine learning models generates a different relevancy-ranked output when the search criterion is applied to the generative AI search system.

12. The non-transitory computer readable medium of claim 11 , wherein causing the generated search criterion to be processed, via a search system, comprises:

performing, by the one or more processors, a first stage scoring process, to generate a set of content items; and

performing, by the one or more processors, a second stage scoring process, by processing the set of content items by an inferencing machine learning model trained to determine item score values; and

generating, by the inferencing machine learning model, item score values for the set of content items.

13. The non-transitory computer readable medium of claim 12 , further comprising the operations of

performing, by the one or more processors, a third stage scoring process that modifies an item score value to adjust a content item position placement in the relevancy-ranked output listing.

14. The non-transitory computer readable medium of claim 12 , further comprising the operations of:

performing, by the one or more processors, a blender process that generates annotations for one or more of the content items in the relevancy-ranked output listing, wherein the generated annotations comprise a type of content item for a respective content item in the relevancy-ranked output listing.

15. The non-transitory computer readable medium of claim 12 , wherein the first stage scoring process comprises:

generating, by the one or more processors, a search vector embedding of the received input;

performing a vector similarity matching of the search vector embedding as to a set of vector embeddings describing the content items; and

generating a listing of those content items where a similarity threshold match value is met or exceeded.

Assignments (3)

Continuity (1)

References Cited (15)

Cited By (2)

US 12,423,313 US 12,688,244