Golden prompt generation based on authoritative publications

1 . A method comprising:

receiving, by data processing hardware, an initial publication associated with a specific topic;

retrieving, by the data processing hardware and using the initial publication, additional publications associated with the specific topic, the initial publication and the additional publications forming a set of publications;

generating, by the data processing hardware and using natural language processing, a set of golden prompts from the set of publications, each golden prompt of the set of golden prompts including text from the set of publications;

generating, by the data processing hardware and based on a pre-trained model, a fine-tuned model by at least weighting each publication from the set of publications to manage an impact each publication has on the fine-tuned model;

generating, by the data processing hardware and using the fine-tuned model and the set of golden prompts, a set of predictions;

determining, by the data processing hardware and using the set of predictions and the set of publications, whether an error rate of the fine-tuned model satisfies a threshold, the error rate indicating a similarity between the set of predictions and the set of publications; and

responsive to determining that the error rate fails to satisfy the threshold:

selecting, by the data processing hardware, a subset of publications from the set of publications by at least removing one or more publications from the set of publications to improve an accuracy of the fine-tuned model;

generating, by the data processing hardware and using natural language processing, a second set of golden prompts from the subset of publications;

generating, by the data processing hardware and based on a second pre-trained model, a second fine-tuned model using the subset of publications;

generating, by the data processing hardware and using the second fine-tuned model and the second set of golden prompts, a second set of predictions; and

determining, by the data processing hardware and using the second set of predictions and the subset of publications, a second error rate of the second fine-tuned model.

2 . The method of claim 1 , wherein the pre-trained model comprises a large language model.

3 . The method of claim 1 , wherein receiving the initial publication comprises receiving, from a user, a request to fine-tune the pre-trained model using the initial publication.

4 . The method of claim 1 , wherein retrieving the additional publications comprises using at least one of a search engine or a web crawler.

5 . The method of claim 1 , wherein retrieving the additional publications comprises searching a public database or searching a private database.

6 . The method of claim 1 , wherein determining the error rate comprises determining an amount of matching between tokens representing the set of predictions to tokens representing the publications.

7 . The method of claim 1 , wherein selecting the subset of publications is based on a credibility of each publication in the set of publications.

8 . The method of claim 1 , further comprising sorting, by the data processing hardware, the set of publications based on a credibility of each publication of the set of publications.

9 . The method of claim 8 , wherein generating a fine-tuned model is based on the credibility of each publication of the set of publications.

10 . A system comprising:

data processing hardware; and

memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to:

receive an initial publication associated with a specific topic;

retrieve, using the initial publication, additional publications associated with the specific topic, the initial publication and the additional publications forming a set of publications;

generate, using natural language processing, a set of golden prompts from the set of publications, each golden prompt of the set of golden prompts including text from the set of publications;

generate, based on a pre-trained model, a fine-tuned model by at least weighting each publication from the set of publications to manage an impact each publication has on the fine-tuned model;

generate, using the fine-tuned model and the set of golden prompts, a set of predictions;

determine, using the set of predictions and the set of publications, whether an error rate of the fine-tuned model satisfies a threshold, the error rate indicating a similarity between the set of predictions and the set of publications; and

responsive to determining that the error rate fails to satisfy the threshold:

select a subset of publications from the set of publications by at least removing one or more publications from the set of publications to improve an accuracy of the fine-tuned model;

generate, using natural language processing, a second set of golden prompts from the subset of publications;

generate, based on a second pre-trained model, a second fine-tuned model using the subset of publications;

generate, using the second fine-tuned model and the second set of golden prompts, a second set of predictions; and

determine, using the second set of predictions and the subset of publications, a second error rate of the second fine-tuned model.

11 . The system of claim 10 , wherein the pre-trained model comprises a large language model.

12 . The system of claim 10 , wherein receiving the initial publication comprises receiving, from a user, a request to fine-tune the pre-trained model using the initial publication.

13 . The system of claim 10 , wherein retrieving the additional publications comprises using at least one of a search engine or a web crawler.

14 . The system of claim 10 , wherein retrieving the additional publications comprises searching a public database or searching a private database.

15 . The system of claim 10 , wherein determining the error rate comprises determining an amount of matching between tokens representing the set of predictions to tokens representing the publications.

16 . The system of claim 10 , wherein selecting the subset of publications is based on a credibility of each publication in the set of publications.

17 . The system of claim 10 , wherein the instructions further cause the data processing hardware to sort the set of publications based on a credibility of each publication of the set of publications.

18 . The system of claim 17 , wherein generating a fine-tuned model is based on the credibility of each publication of the set of publications.