Diversity aware media content recommendation
A reinforcement learning ranker can take into account previously-recommended media content items to produce a ranked list of media content items to recommend next. The ranker finds a policy that gives the probability of sampling a media content item given a state. The policy is learned such that it maximizes a reward. A reward function associated with the media content item can be defined with respect to whether the user finds the media content item relevant (likelihood that the user will like the media content item) and a diversity score of the media content item.
1 . A method for selecting a media content item, the method comprising:
obtaining first embedding data describing media content items from previous content consumption sessions of a user account, wherein the first embedding data is based on a weighting of session representations from the previous content consumption sessions;
obtaining second embedding data regarding media content items previously recommended during a current content consumption session of the user account;
obtaining third embedding data describing a potential media content item;
providing, to a reinforcement learning model, the first embedding data, the second embedding data, and the third embedding data, wherein the reinforcement learning model produces a score, wherein the score is based on: a relevance of features of the potential media content item with respect to the user account, and a diversity of the potential media content item with respect to the user account, and wherein the diversity is weighted by the relevance and a weighting factor; and
selecting, for the user account, the potential media content item based on the score, wherein the potential media content item is selected for playback to a device associated with the user account.
2 . The method of claim 1 , wherein the potential media content item is an audio track.
3 . The method of claim 1 , wherein the diversity is based on a dissimilarity of the potential media content item with respect to media content items from the previous content consumption sessions.
4 . A non-transitory computer-readable medium having instructions stored thereon that, when executed by one or more processors, cause the one or more processors to:
obtain first embedding data describing media content items from previous content consumption sessions of a user account, wherein the first embedding data is based on a weighting of session representations from the previous content consumption sessions;
obtain second embedding data regarding media content items previously recommended during a current content consumption session of the user account;
obtain third embedding data describing a potential media content item;
provide, to a reinforcement learning model, the first embedding data, the second embedding data, and the third embedding data, wherein the reinforcement learning model produces a score, wherein the score is based on: a relevance of features of the potential media content item with respect to the user account, and a diversity of the potential media content item with respect to the user account, and wherein the diversity is weighted by the relevance and a weighting factor; and
select, for the user account, the potential media content item based on the score, wherein the potential media content item is selected for playback to a device associated with the user account.
5 . A media-delivery system comprising:
one or more processors;
memory; and
program instructions, stored in the memory, that upon execution by the one or more processors cause the media-delivery system to perform operations comprising:
obtain first embedding data describing media content items from previous content consumption sessions of a user account, wherein the first embedding data is based on a weighting of session representations from the previous content consumption sessions;
obtain second embedding data regarding media content items previously recommended during a current content consumption session of the user account;
obtain third embedding data describing a potential media content item;
provide, to a reinforcement learning model, the first embedding data, the second embedding data, and the third embedding data, wherein the reinforcement learning model produces a score, wherein the score is based on: a relevance of features of the potential media content item with respect to the user account, and a diversity of the potential media content item with respect to the user account, and wherein the diversity is weighted by the relevance and a weighting factor;
select, for the user account, the potential media content item based on the score; and
transmit the potential media content item to a media-playback device for playback.
6 . The media-delivery system of claim 5 , wherein the diversity is based on a dissimilarity of the potential media content item with respect to media content items from the previous content consumption sessions.
7 . The method of claim 1 , wherein the reinforcement learning model applies a long short-term memory (LSTM) layer to the first embedding data and the second embedding data to produce an output, and wherein the reinforcement learning model applies a softmax function to a concatenation of the output and the third embedding data to produce the score.
8 . The method of claim 7 , wherein the LSTM layer comprises a stacked LSTM initialized based on a session meta feature relating to a combination of the first embedding data and data related to the user account.
9 . The non-transitory computer-readable medium of claim 4 , wherein the reinforcement learning model applies a long short-term memory (LSTM) layer to the first embedding data and the second embedding data to produce an output, and wherein the reinforcement learning model applies a softmax function to a concatenation of the output and the third embedding data to produce the score.
10 . The non-transitory computer-readable medium of claim 9 , wherein the LSTM layer comprises a stacked LSTM initialized based on a session meta feature relating to a combination of the first embedding data and data related to the user account.
11 . The media-delivery system of claim 5 , wherein the reinforcement learning model applies a long short-term memory (LSTM) layer to the first embedding data and the second embedding data to produce an output, and wherein the reinforcement learning model applies a softmax function to a concatenation of the output and the third embedding data to produce the score.
12 . The media-delivery system of claim 11 , wherein the LSTM layer comprises a stacked LSTM initialized based on a session meta feature relating to a combination of the first embedding data and data related to the user account.
13 . The method of claim 1 , further comprising:
transmitting the potential media content item to the device for the playback.
14 . The non-transitory computer-readable medium of claim 4 , wherein the instructions, when executed by the one or more processors, further cause the one or more processors to:
transmit the potential media content item to the device for the playback.