Systems and methods for generating a self-learning and self-improving dialogue system

1 . A computer-implemented method of generating a self-learning and a self-improving dialogue system, the method comprising:

receiving a first language model trained to emulate a persona of a first user of the dialogue system;

receiving a second language model trained to evaluate a first dialogue system output;

generating, using the first language model, a first language model output emulating the persona of the first user of the dialogue system;

providing the first language model output to a third language model, wherein the third language model is trained to generate one or more dialogue system outputs from the dialogue system;

responsive to the first language model output, generating, using the third language model, the first dialogue system output, thereby generating a simulated dialogue between the first language model and the third language model, the simulated dialogue comprising at least the first language model output and the first dialogue system output;

evaluating, using the second language model, the first dialogue system output;

fine-tuning, using the generated simulated dialogue and the evaluation of the first dialogue system output, the third language model; and

fine-tuning, based at least in part on a second dialogue system output from the dialogue system and a feedback on the second dialogue system output, the second language model.

2 . The method of claim 1 , wherein the first language model is trained using supervised fine-tuning and based on one or more dialogues between the dialogue system and the first user of the dialogue system, each dialogue of the one or more dialogues comprising at least one dialogue system input from the first user and at least one dialogue system output from the dialogue system to the first user.

3 . The method of claim 1 , wherein the first language model is updated at a predetermined time intervals.

4 . The method of claim 1 , wherein generating the first language model output comprises providing the first language model with a first language model prompt, wherein the first language model prompt includes instructions to guide the first language model to generate one or more first language model outputs based on a task, a scenario, or both.

5 . The method of claim 1 , wherein the second language model is trained using supervised fine-tuning and based on: a plurality of dialogue system outputs from the dialogue system, and feedback from a human manager to each of the plurality of dialogue system outputs.

6 . The method of claim 1 , wherein evaluating the first dialogue system output further comprises:

responsive to the first language model output, generating, using the third language model, the first dialogue system output and a third dialogue system output;

comparing, using the second language model, the first dialogue system output and the third dialogue system output; and

identifying the first dialogue system output as a preferable dialogue system output of the first dialogue system output and the third dialogue system output.

7 . The method of claim 6 , further comprising:

annotating, using the second language model, the first dialogue system output as a preferable dialogue system output, thereby generating an annotated first dialogue system output; and

annotating, using the second language model, the third dialogue system output as a not preferable dialogue system output, thereby generating an annotated third dialogue system output,

wherein fine-tuning the third language model comprises providing the third language model with the annotated first dialogue system output and the annotated third dialogue system output.

8 . The method of claim 1 further comprising:

receiving a second input from a second user of the dialogue system;

responsive to second input, generating, using the fine-tuned third language model, a third dialogue system output.

9 . The method of claim 8 , further comprising:

responsive to second input, generating, using the fine-tuned third language model, the third dialogue system output and a fourth dialogue system output;

evaluating, using the second language model, the third dialogue system output and the fourth dialogue system output;

annotating the third dialogue system output as a preferable dialogue system output of the third dialogue system output and the fourth dialogue system output; and

outputting the third dialogue system output to the second user.

10 . The method of claim 1 , wherein fine-tuning the third language model comprises fine-tuning based on direct preference optimisation.

11 . A non-transitory computer readable storage medium comprising computer readable code configured to cause a computer to perform a method comprising the following operations:

receiving a first language model trained to emulate a persona of a first user of a dialogue system;

receiving a second language model trained to evaluate a first dialogue system output;

generating, using the first language model, a first language model output emulating the persona of the first user of the dialogue system;

providing the first language model output to a third language model, wherein the third language model is trained to generate one or more dialogue system outputs from the dialogue system;

responsive to the first language model output, generating, using the third language model, the first dialogue system output, thereby generating a simulated dialogue between the first language model and the third language model, the simulated dialogue comprising at least the first language model output and the first dialogue system output;

evaluating, using the second language model, the first dialogue system output;

fine-tuning, using the generated simulated dialogue and the evaluation of the first dialogue system output, the third language model; and

fine-tuning, based at least in part on a second dialogue system output from the dialogue system and a feedback on the second dialogue system output, the second language model.

12 . The non-transitory computer readable storage medium of claim 11 , the method comprising further operations:

training, using supervised fine-tuning and based on one or more dialogues between the dialogue system and the first user of the dialogue system, the first language model, each dialogue of the one or more dialogues comprising at least one dialogue system input from the first user and at least one dialogue system output from the dialogue system to the first user.

13 . The non-transitory computer readable storage medium of claim 11 , the method comprising further operations:

training, using supervised fine-tuning, the second language model, wherein the training is based on: a plurality of dialogue system outputs from the dialogue system, and feedback from a human manager to each of the plurality of dialogue system outputs.

14 . The non-transitory computer readable storage medium of claim 11 , the method comprising further operations:

responsive to the first language model output, generating, using the third language model, the first dialogue system output and a third dialogue system output;

comparing, using the second language model, the first dialogue system output and the third dialogue system output; and

identifying the first dialogue system output as a preferable dialogue system output of the first dialogue system output and the third dialogue system output.

15 . The non-transitory computer readable storage medium of claim 14 , the method comprising further operations:

annotating, using the second language model, the first dialogue system output as a preferable dialogue system output, thereby generating an annotated first dialogue system output; and

annotating, using the second language model, the third dialogue system output as a not preferable dialogue system output, thereby generating an annotated third dialogue system output,

wherein fine-tuning the third language model comprises providing the third language model with the annotated first dialogue system output and the annotated third dialogue system output.

16 . A dialogue system, comprising:

one or more processors, configured to:

receive a first language model trained to emulate a persona of a first user of the dialogue system;

receive a second language model trained to evaluate a first dialogue system output;

generate, using the first language model, a first language model output emulating the persona of the first user of the dialogue system;

provide the first language model output to a third language model, wherein the third language model is trained to generate one or more dialogue system outputs from the dialogue system;

responsive to the first language model output, generate, using the third language model, the first dialogue system output, thereby generating a simulated dialogue between the first language model and the third language model, the simulated dialogue comprising at least the first language model output and the first dialogue system output;

evaluate, using the second language model, the first dialogue system output;

fine-tune, using the generated simulated dialogue and the evaluation of the first dialogue system output, the third language model; and

fine-tuning, based at least in part on a second dialogue system output from the dialogue system and a feedback on the second dialogue system output, the second language model.