Method for automatic generation of frequently asked questions
Methods and systems for generating a frequently asked questions are provided, which include defining, by a computer program executed by a computer, a first large language model (LLM) with a user query and a feedback of the user query; refining, by the computer program, the user query to a question set based on the feedback, the question set comprising one or more sentences; defining, by the computer program, a second LLM to generate a first set of question and answer pairs from a source document; defining, by the computer program, a third LLM to generate a content set from the source document based on a rewriting of the source document; selecting, by the computer program, top questions from the content set to be provided to the second LLM; and generating, by the second LLM, a second set of question and answer pairs based on the top questions.
1 . A method comprising:
inputting, by a computer program executed by a computer, a user query set into a first large language model (LLM);
refining the user query set into a first question set based on a feedback criteria comprising an existence of a question word in a first question;
inputting, by the computer program, a source document to a second LLM, the second LLM configured to generate a content set from the source document, wherein the source document is selected based on similarity to a content of the user query;
selecting, by the computer program, a second question set as a subset of the first question set based on a semantic similarity to the content set, wherein the top semantic similarity is determined based on an embedding-wise cosine similarity score;
generating, by a third LLM, a set of question and answer pairs based on the second question set provided by the computer program;
selecting, by the computer program, a subset of the set of question and answer pairs based on a semantic similarity to the first question set;
linking, by the computer program, the subset of the set of question and answer pairs to the source document, wherein the computer program saves a link to the source document in the cache to a database;
publishing, by the computer program, the subset of the set of question and answer pairs;
saving, by the computer program, the source document to a cache;
periodically checking, by the computer program, changes to the source document;
providing, by the computer program, the changes to the second LLM to generate a revised content set;
providing, by the computer program, the revised content to the third LLM;
generating, by the third LLM, a revised set of question and answer pairs; and
deleting and replacing, by the computer program, the subset of the set of question and answer pairs with the revised set of question and answer pairs.
2 . A computer processing system comprising:
a memory configured to store instructions; and
a hardware processor operatively coupled to the memory for executing the instructions of a text or call processing program to:
input, by a computer program executed by a computer, a user query set into a first large language model (LLM);
refine the user query set into a first question set based on a feedback criteria comprising an existence of a question word in a question word in the first question;
input, by the computer program, a source document to a second LLM, the second LLM configured to generate a content set from the source document, wherein the source document is selected based on similarity to a content of the user query;
select, by the computer program, a second question set as a subset of the first question set based on a semantic similarity to the content set;
generate, by a third LLM, a set of question and answer pairs based on the second question set provided by the computer program;
select, by the computer program, a subset of the set of question and answer pairs based on a semantic similarity to the first question set, wherein the top semantic similarity is determined based on an embedding-wise cosine similarity score;
link, by the computer program, the subset of the set of question and answer pairs to the source document, wherein the computer program saves a link to the source document in the cache to a database; and
publish, by the computer program, the subset of the set of question and answer pairs;
save, by the computer program, the source document to a cache;
periodically check, by the computer program, changes to the source document;
provide, by the computer program, the changes to the second LLM to generate a revised content set;
provide, by the computer program, the revised content to the third LLM;
generate, by the third LLM, a revised set of question and answer pairs; and
delete and replace, by the computer program, the subset of the set of question and answer pairs with the revised set of question and answer pairs.
3 . A non-transitory computer readable storage medium, including instructions stored thereon, which when read and executed by one or more computer processors, cause the one or more computer processors to perform steps comprising:
inputting, by a computer program executed by a computer, a user query set into a first large language model (LLM);
refining the user query set into a first question set based on a feedback criteria comprising an existence of a question word in a question word in the first question;
inputting, by the computer program, a source document to a second LLM, the second LLM configured to generate a content set from the source document, wherein the source document is selected based on similarity to a content of the user query;
selecting, by the computer program, a second question set as a subset of the first question set based on a semantic similarity to the content set;
generating, by a third LLM, a set of question and answer pairs based on the second question set provided by the computer program;
selecting, by the computer program, a subset of the set of question and answer pairs based on a semantic similarity to the first question set, wherein the top semantic similarity is determined based on an embedding-wise cosine similarity score;
linking, by the computer program, the subset of the set of question and answer pairs to the source document, wherein the computer program saves a link to the source document in the cache to a database;
publishing, by the computer program, the subset of the set of question and answer pairs;
saving, by the computer program, the source document to a cache;
periodically checking, by the computer program, changes to the source document;
providing, by the computer program, the changes to the second LLM to generate a revised content set;
providing, by the computer program, the revised content to the third LLM;
generating, by the third LLM, a revised set of question and answer pairs; and
deleting and replacing, by the computer program, the subset of the set of question and answer pairs with the revised set of question and answer pairs.