Interest-based conversational recommendation system
Disclosed herein are system, method and/or computer program product embodiments, and/or combinations thereof, for training a conversational recommendation system. An embodiment generates a probabilistic pseudo-user neural network model based on at least one interest probability distribution corresponding to a pseudo-user profile. The embodiment trains, using the pseudo-user neural network model, the conversational recommendation system to learn a recommendation policy, where the conversational recommendation system includes an interest-exploration engine and a prompt-decision engine. The training includes performing an iterative learning process that includes selecting an interest-exploration strategy based on one or more of the following: an interest-exploration policy, an earlier pseudo-user response generated by the pseudo-user neural network model, content data, and pseudo-user interaction history. The embodiment then generates, using the trained conversational recommendation system, a real-time recommendation having high play probability based on the minimal number of iterations of conversation between a user and the trained conversational recommendation system.
1. A computer-implemented method for training a conversational recommendation system for generating an output, having a high play-probability, based on a minimal number of iterations of conversation, comprising:
generating, by at least one computer processor, a probabilistic pseudo-user neural network model based on at least one interest probability distribution corresponding to a pseudo-user profile;
training, using the probabilistic pseudo-user neural network model, the conversational recommendation system to learn a recommendation policy, wherein the conversational recommendation system comprises an interest-exploration engine and a prompt-decision engine, and wherein the training includes performing one or more iterations of an iterative learning process, including:
selecting, by the interest-exploration engine, an interest-exploration strategy based on one or more of the following: an interest-exploration policy, an earlier pseudo-user response generated by the probabilistic pseudo-user neural network model, content data, and pseudo-user interaction history;
selecting, by the prompt-decision engine, an interest prompt based on a prompt-decision policy and the selected interest-exploration strategy;
generating, by the probabilistic pseudo-user neural network model, another pseudo-user response based on the selected interest prompt;
updating a reward function, corresponding to the interest-exploration engine and the prompt-decision engine, based on the another pseudo-user response; and
updating, using a reinforcement-learning method, the interest-exploration policy and the prompt-decision policy based on at least the updated reward function; and
generating, using the trained conversational recommendation system, a real-time recommendation having the high play-probability based on the minimal number of iterations of conversation between a user and the trained conversational recommendation system.
2. The method of claim 1 , further comprising:
terminating the iterative learning process if the another pseudo-user response comprises accepting to play a recommended media content corresponding to the selected interest prompt.
3. The method of claim 1 , wherein the recommendation policy corresponds to the interest-exploration policy and the prompt-decision policy that cumulatively result in generating a pseudo-user response of accepting to play a recommended media content in a minimal number of iterations of the iterative learning process.
4. The method of claim 1 , wherein the at least one interest probability distribution further comprises a long-term interest probability distribution and a short-term interest probability distribution corresponding to the pseudo-user profile.
5. The method of claim 1 , wherein the another pseudo-user response comprises accepting to play a recommended media content corresponding to the selected interest prompt, quitting a conversation session with the conversational recommendation system, or generating a further pseudo-user response.
6. The method of claim 5 , wherein the updating the reward function further comprises:
incrementing the reward function by a predetermined value if the another pseudo-user response comprises accepting to play the recommended media content corresponding to the selected interest prompt;
decrementing the reward function by a first value if the another pseudo-user response comprises quitting the conversation session with the conversational recommendation system; and
decrementing the reward function by a second value if the another pseudo-user response comprises generating the further pseudo-user response.
7. The method of claim 1 , wherein the selecting the interest-exploration strategy further comprises:
extracting a current interest from the pseudo-user interaction history using named entity recognition; and
performing an interest prediction based on the current interest.
8. The method of claim 1 , wherein the selecting the interest-exploration strategy further comprises:
selecting the interest-exploration strategy from a plurality of candidate interest-exploration strategies, including one or more of the following: exploration via an area target, exploration via a point target, exploration via a filtered target, exploration via a popular target, and exploration via a similar target.
9. The method of claim 1 , wherein a response generated by the probabilistic pseudo-user neural network model is processed by an automatic speech recognition module and a natural language understanding module before being received by the interest exploration engine.
10. The method of claim 1 , wherein an output of the prompt decision engine corresponding to the selected interest prompt is processed by a large language model and a text to speech module before being received by the probabilistic pseudo-user neural network model.
11. A system, comprising:
one or more memories; and
at least one processor each coupled to at least one of the memories and configured to perform operations comprising:
generating a probabilistic pseudo-user neural network model based on at least one interest probability distribution corresponding to a pseudo-user profile;
training, using the probabilistic pseudo-user neural network model, a conversational recommendation system to learn a recommendation policy, wherein the conversational recommendation system comprises an interest-exploration engine and a prompt-decision engine, and wherein the training includes performing one or more iterations of an iterative learning process, including:
selecting, by the interest-exploration engine, an interest-exploration strategy based on one or more of the following: an interest-exploration policy, an earlier pseudo-user response generated by the probabilistic pseudo-user neural network model, content data, and pseudo-user interaction history;
selecting, by the prompt-decision engine, an interest prompt based on a prompt-decision policy and the selected interest-exploration strategy;
generating, by the probabilistic pseudo-user neural network model, another pseudo-user response based on the selected interest prompt;
updating a reward function, corresponding to the interest-exploration engine and the prompt-decision engine, based on the another pseudo-user response; and
updating, using a reinforcement-learning method, the interest-exploration policy and the prompt-decision policy based on at least the updated reward function; and
generating, using the trained conversational recommendation system, a real-time recommendation having a high play-probability based on a minimal number of iterations of conversation between a user and the trained conversational recommendation system.
12. The system of claim 11 , the operations further comprising:
terminating the iterative learning process if the another pseudo-user response comprises accepting to play a recommended media content corresponding to the selected interest prompt.
13. The system of claim 11 , wherein the recommendation policy corresponds to the interest-exploration policy and the prompt-decision policy that cumulatively result in generating a pseudo-user response of accepting to play a recommended media content in a minimal number of iterations of the iterative learning process.
14. The system of claim 11 , wherein the at least one interest probability distribution further comprises a long-term interest probability distribution and a short-term interest probability distribution corresponding to the pseudo-user profile.
15. The system of claim 11 , wherein the another pseudo-user response comprises accepting to play a recommended media content corresponding to the selected interest prompt, quitting a conversation session with the conversational recommendation system, or generating a further pseudo-user response.
16. A non-transitory computer-readable medium having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising:
generating, by at least one computer processor, a probabilistic pseudo-user neural network model based on at least one interest probability distribution corresponding to a pseudo-user profile;
training, using the probabilistic pseudo-user neural network model, a conversational recommendation system to learn a recommendation policy, wherein the conversational recommendation system comprises an interest-exploration engine and a prompt-decision engine, and wherein the training includes performing one or more iterations of an iterative learning process, including:
selecting, by the interest-exploration engine, an interest-exploration strategy based on one or more of the following: an interest-exploration policy, an earlier pseudo-user response generated by the probabilistic pseudo-user neural network model, content data, and pseudo-user interaction history;
selecting, by the prompt-decision engine, an interest prompt based on a prompt-decision policy and the selected interest-exploration strategy;
generating, by the probabilistic pseudo-user neural network model, another pseudo-user response based on the selected interest prompt;
updating a reward function, corresponding to the interest-exploration engine and the prompt-decision engine, based on the another pseudo-user response; and
updating, using a reinforcement-learning method, the interest-exploration policy and the prompt-decision policy based on at least the updated reward function; and
generating, using the trained conversational recommendation system, a real-time recommendation having a high play-probability based on a minimal number of iterations of conversation between a user and the trained conversational recommendation system.
17. The non-transitory computer-readable medium of claim 16 , the operations further comprising:
terminating the iterative learning process if the another pseudo-user response comprises accepting to play a recommended media content corresponding to the selected interest prompt.
18. The non-transitory computer-readable medium of claim 16 , wherein the recommendation policy corresponds to the interest-exploration policy and the prompt-decision policy that cumulatively result in generating a pseudo-user response of accepting to play a recommended media content in a minimal number of iterations of the iterative learning process.
19. The non-transitory computer-readable medium of claim 16 , wherein the at least one interest probability distribution further comprises a long-term interest probability distribution and a short-term interest probability distribution corresponding to the pseudo-user profile.
20. The non-transitory computer-readable medium of claim 16 , wherein the another pseudo-user response comprises accepting to play a recommended media content corresponding to the selected interest prompt, quitting a conversation session with the conversational recommendation system, or generating a further pseudo-user response.