Methods and systems for personalized screen content optimization
Systems and associated methods are described for providing content recommendations. The system selects, using a multi-armed bandit solution model, a first plurality of content categories based on a reward score of each content category. The categories are displayed. When a user selects an item from the displayed categories, the system finds all categories that include the selected item, but rewards only the category with the highest score. The system selects, using the multi-armed bandit solution model, the second plurality of content categories based on the updated reward score of each content category. The categories are then displayed. The system may also repeat the steps to refine the multi-armed bandit solution model.
1 . A method comprising:
selecting, by a server executing a content recommendation application, based on demographical data of a user, a first plurality of applications, from a plurality of applications, that are associated with a first plurality of categories selected by the recommendation application using a multi-armed bandit selection algorithm, from a plurality of categories, for recommendation on a device, wherein each category of the first plurality of content categories is associated with a reward score;
receiving, at the server, from the device a request to access an application from the first plurality of applications;
identifying, by the server, multiple categories, from the first plurality of categories, that include the requested application;
only increasing, by the server, reward score of one content category, from the multiple identified categories, that have the highest reward score;
tracking, by the server, the reward score of only those content categories whose reward score is increased to determine whether the reward score continues to increase over a predetermined period; and
in response to determining that the reward score does not continue to increase:
using the increased reward score, which includes any continued increases, to select a second plurality of content categories based on the increased reward score of each multiple identified categories using the multi-armed bandit selection algorithm;
selecting, based on demographical data of the user, a second plurality of applications from the second plurality of content categories; and
causing the device to generate for display, identifiers of the second plurality of applications, wherein the display of the identifiers of the second plurality of applications is generated based on instructions transmitted by the server to the device.
2 . The method of claim 1 , wherein, determining that the reward score does not continue to increase comprises, determining that the increase in reward score over the predetermined period is negative, zero, or below a predetermined positive number.
3 . The method of claim 1 , wherein determining that the reward score does not continue to increase over the predetermined prior of time comprises:
maintaining a counter to track a number of iterations of selections of the content categories whose reward score is increased; and
continuing to track the number of iterations until a predetermined number of iterations are reached.
4 . The method of claim 3 , further comprising, continuing to track the reward score after the predetermined number of iterations are reached until a determination is made that the reward score does not continue to increase.
5 . The method of claim 3 , further comprising, tracking a history of rewards and using data related to history of rewards and number of iterations in determining the second plurality of application from the second plurality of content categories.
6 . The method of claim 3 , further comprising, switching to an exploitation stage upon determining that the reward score does not continue to increase.
7 . The method of claim 6 , wherein the exploitation stage relates to selecting the second plurality of applications from the second plurality of content categories based on which content categories have the highest reward score after a predetermined number of iterations are reached.
8 . The method of claim 1 , further comprising, in response to determining that the reward score continues to increase, selecting the second plurality of applications from the second plurality of content categories based on a random technique.
9 . The method of claim 1 , further comprising:
switching from an exploration stage to an exploitation stage upon determining that the reward score does not continue to increase over the predetermined period of time.
10 . The method of claim 1 , further comprising, grouping the first plurality of applications in a group based on their association with the first plurality of categories.
11 . A system comprising:
communications circuitry configured to access a database containing content items; and
control circuitry configured to:
select, based on demographical data of a user, a first plurality of applications, from a plurality of applications, that are associated with a first plurality of categories selected by a recommendation application using a multi-armed bandit selection algorithm, from a plurality of categories, for recommendation on a device, wherein each category of the first plurality of content categories is associated with a reward score;
receive from the device a request to access an application from the first plurality of applications;
identify multiple categories, from the first plurality of categories, that include the requested application;
only increase reward score of one content category, from the multiple identified categories, that have the highest reward score;
track the reward score of only those content categories whose reward score is increased to determine whether the reward score continues to increase over a predetermined period; and
in response to determining that the reward score does not continue to increase:
use the increased reward score, which includes any continued increases, to select a second plurality of content categories based on the increased reward score of each multiple identified categories using the multi-armed bandit selection algorithm;
select, based on demographical data of the user, a second plurality of applications from the second plurality of content categories; and
cause the device to generate for display, identifiers of the second plurality of applications, wherein the display of the identifiers of the second plurality of applications is generated based on instructions transmitted by the server to the device.
12 . The system of claim 11 , further comprising, the control circuitry configured to switch from an exploration stage to an exploitation stage upon determining that the reward score does not continue to increase over the predetermined period of time.
13 . The system of claim 11 , wherein determining that the reward score does not continue to increase over the predetermined prior of time comprises, the control circuitry configured to:
maintain a counter to track a number of iterations of selections of the content categories whose reward score is increased; and
continue to track the number of iterations until a predetermined number of iterations are reached.
14 . The system of claim 13 , further comprising, the control circuitry configured to track a history of rewards and using data related to history of rewards and number of iterations in determining the second plurality of application from the second plurality of content categories.
15 . The system of claim 13 , further comprising, the control circuitry configured to continue tracking the reward score after the predetermined number of iterations are reached until a determination is made that the reward score does not continue to increase.
16 . The system of claim 11 , further comprising, the control circuitry configured to switch to an exploitation stage upon determining that the reward score does not continue to increase.
17 . The system of claim 16 , wherein the exploitation stage relates to the control circuitry configured to select the second plurality of applications from the second plurality of content categories based on which content categories have the highest reward score after the predetermined number of iterations are reached.
18 . The system of claim 11 , further comprising, in response to determining that the reward score continues to increase, the control circuitry configured to select the second plurality of applications from the second plurality of content categories based on a random technique.
19 . The system of claim 11 , wherein, determining that the reward score does not continue to increase comprises, the control circuitry configured to determine that the increase in reward score over the predetermined period is negative, zero, or below a predetermined positive number.
20 . The system of claim 19 , further comprising, the control circuitry configured to group the first plurality of applications in a group based on their association with the first plurality of categories.