Personalized model threshold
In an example the output of a machine learned model is a score is then compared to a threshold, and if the score transgresses the threshold, the corresponding item is available to be recommended to the user via the graphical user interface. In an example embodiment, rather than a fixed (static) threshold, a dynamic threshold is utilized. This dynamic threshold is based on a harmonic mean of probabilities utilized in the GLMix model. Specifically, the GLMix model may calculate and utilize the probability that a user will engage with a particular item via a graphical user interface, and also a probability that a user will dismiss a particular item via a graphical user interface.
1. A system comprising:
a computer-readable medium having instructions stored thereon, which, when executed by a processor, cause the system to perform operations comprising:
obtaining a first set of features derived from attributes of a first user in a social networking service;
obtaining a second set of features derived from activity of the first user, with respect to a plurality of items in the social networking service;
for each of a plurality of items considered for display in a graphical user interface of the social networking service:
obtaining a third set of features derived from attributes of the item;
feeding the first, second, and third set of features into an engagement model trained to output an engagement score indicating a likelihood that the first user will positively engage with the item;
feeding the first, second, and third set of features into a dismiss model trained to output a dismiss score indicating a likelihood that the first user will negatively engage with the item;
feeding the engagement score and dismiss score into an item scoring model trained to output an item score;
using the engagement score and dismiss score to calculate a dynamic threshold;
comparing the item score to the dynamic threshold;
in response to a determination that the item score does not transgress the dynamic threshold, eliminating the item from consideration for display in the graphical user interface of the social networking service;
ranking the items considered for display in the graphical user interface based on their respective item scores; and
displaying one or more of the items considered for display in the graphical user interface based on the ranking.
2. The system of claim 1 , wherein the item scoring model is a Generalized Linear Mixed Effect (GLMix) mode.
3. The system of claim 1 , wherein the operations further comprise:
training the engagement model by feeding training data into a first machine learning algorithm, the training data having information about which items a plurality of users have positively engaged with; and
training the dismiss model by feeding training data into a second machine learning algorithm, the training data having information about which items a plurality of users have negatively engaged with.
4. The system of claim 3 , herein the first and second machine learning algorithms are identical.
5. The system of claim 4 , herein the first and second machine learning algorithms are generalized linear mixed effect (GLMix) machine learning algorithms.
6. The system of claim 1 , wherein the harmonic mean is equal to
2
*
p
engage
*
(
1
-
p
dismiss
)
p
engage
+
(
1
-
p
dismiss
)
,
wherein p engage is the engagement score and p dismiss is the dismiss score.
7. The system of claim 1 , wherein the items are job postings listed in the social networking service.
8. A computerized method comprising:
obtaining a first set of features derived from attributes of a first user in a social networking service;
obtaining a second set of features derived from activity of the first user, with respect to a plurality of items in the social networking service;
for each of a plurality of items considered for display in a graphical user interface of the social networking service:
obtaining a third set of features derived from attributes of the item;
feeding the first, second, and third set of features into an engagement model trained to output an engagement score indicating a likelihood that the first user will positively engage with the item;
feeding the first, second, and third set of features into a dismiss model trained to output a dismiss score indicating a likelihood that the first user will negatively engage with the item;
feeding the engagement score and dismiss score into an item scoring model trained to output an item score;
using the engagement score and dismiss score to calculate a dynamic threshold;
comparing the item score to the dynamic threshold;
in response to a determination that the item score does not transgress the dynamic threshold, eliminating the item from consideration for display in the graphical user interface of the social networking service;
ranking the items considered for display in the graphical user interface based on their respective item scores; and
displaying one or more of the items considered for display in the graphical user interface based on the ranking.
9. The method of claim 8 , wherein the item scoring model is a Generalized Linear Mixed Effect (GLMix) mode.
10. The method of claim 8 , further comprising:
training the engagement model by feeding training data into a first machine learning algorithm, the training data having information about which items a plurality of users have positively engaged with; and
training the dismiss model by feeding training data into a second machine learning algorithm, the training data having information about which items a plurality of users have negatively engaged with.
11. The method of claim 10 , herein the first and second machine learning algorithms are identical.
12. The method of claim 11 , herein the first and second machine learning algorithms are generalized linear mixed effect (GLMix) machine learning algorithms.
13. The method of claim 8 , wherein the harmonic mean is equal to
2
*
p
engage
*
(
1
-
p
dismiss
)
p
engage
+
(
1
-
p
dismiss
)
,
wherein p engage is the engagement score and p dismiss is the dismiss score.
14. The method of claim 8 , wherein the items are job postings listed in the social networking service.
15. A non-transitory machine-readable storage medium comprising instructions which, when implemented by one or more machines, cause the one or more machines to perform operations comprising:
obtaining a first set of features derived from attributes of a first user in a social networking service;
obtaining a second set of features derived from activity of the first user, with respect to a plurality of items in the social networking service;
for each of a plurality of items considered for display in a graphical user interface of the social networking service:
obtaining a third set of features derived from attributes of the item;
feeding the first, second, and third set of features into an engagement model trained to output an engagement score indicating a likelihood that the first user will positively engage with the item;
feeding the first, second, and third set of features into a dismiss model trained to output a dismiss score indicating a likelihood that the first user will negatively engage with the item;
feeding the engagement score and dismiss score into an item scoring model trained to output an item score;
using the engagement score and dismiss score to calculate a dynamic threshold;
comparing the item score to the dynamic threshold;
in response to a determination that the item score does not transgress the dynamic threshold, eliminating the item from consideration for display in the graphical user interface of the social networking service;
ranking the items considered for display in the graphical user interface based on their respective item scores; and
displaying one or more of the items considered for display in the graphical user interface based on the ranking.
16. The non-transitory machine-readable storage medium of claim 15 , wherein the item scoring model is a Generalized Linear Mixed Effect (GLMix) mode.
17. The non-transitory machine-readable storage medium of claim 15 , wherein the operations further comprise:
training the engagement model by feeding training data into a first machine learning algorithm, the training data having information about which items a plurality of users have positively engaged with; and
training the dismiss model by feeding training data into a second machine learning algorithm, the training data having information about which items a plurality of users have negatively engaged with.
18. The non-transitory machine-readable storage medium of claim 17 , wherein the first and second machine learning algorithms are identical.
19. The non-transitory machine-readable storage medium of claim 18 , wherein the first and second machine learning algorithms are generalized linear mixed effect (GLMix) machine learning algorithms.
20. The non-transitory machine-readable storage medium of claim 15 , wherein the harmonic mean is equal to
2
*
p
engage
*
(
1
-
p
dismiss
)
p
engage
+
(
1
-
p
dismiss
)
,
wherein p engage is the engagement score and p dismiss is the dismiss score.