Using long-term features and short-term features to filter content before a selection process to reduce latency of a content distribution system
Using long-term features and short-term features to reduce latency in providing recommendations is described. A user device in a session with an online system may request a recommendation. The online system identifies a set of recommendations based in part on the request. The online system retrieves long-term features for each of the set, and determines short-term features for each of the set. The short-term features are based on the session. The online system applies the long-term features and the short-term features to a scoring model that scores the recommendations of the set. The online system selects a subset of the set based on the scores, and provides the selected subset to a selector that selects a recommendation from the subset. The recommendation may be provided to the user device prior to expiration of a latency period associated with serving the recommendation request.
1 . A method, performed at a computer system comprising a processor and a computer-readable medium, comprising:
responsive to a page load event that is part of a session between the computer system and a user device of a user, identifying a set of recommendations, wherein the page load event is associated with a target latency period;
retrieving long-term features for each of the set of recommendations, wherein each of the long-term features describes a relationship between the user and a recommendation of the set of recommendations;
generating short-term features for each of the set of recommendations, wherein each of the short-term features describes a relationship between the session and a recommendation of the set of recommendations;
applying the long-term features and the short-term features to a scoring model that outputs scores for each of the set of recommendations;
selecting a set of candidate recommendations from the set of recommendations based in part on the scores;
providing the selected candidate recommendations to a selector, wherein the selector selects one or more of the candidate recommendations; and
providing the selected one or more candidate recommendations to the user device before expiration of the target latency period, wherein providing the selected one or more candidate recommendations to the user device causes the user device to display the one or more candidate recommendations.
2 . The method of claim 1 , wherein applying the long-term features and the short-term features to the scoring model comprises:
identifying a platform context of the user device; and
applying the long-term features, the short-term features, and the platform context to the scoring model that outputs the scores for the set.
3 . The method of claim 1 , wherein applying the long-term features and the short-term features to the scoring model comprises:
identifying a recommendation slot context for an ordering interface of the user device; and
applying the long-term features, the short-term features, and the recommendation slot context to the scoring model that outputs the scores for the set,
wherein the recommendation provided to the user device is presented via the ordering interface.
4 . The method of claim 1 , wherein generating the short-term features for each of the set of recommendations comprises:
using a session model to generate session scores for each of the set of recommendations, wherein the generated session scores are the short-term features.
5 . The method of claim 1 , further comprising:
before the session, generating relevance scores for each of the set using user embeddings and item embeddings, wherein retrieving the long-term features comprises retrieving the generated relevance scores.
6 . The method of claim 1 , further comprising:
before the session, generating conversion probability scores for the set, wherein retrieving the long-term features comprises retrieving the generated conversion probability scores.
7 . The method of claim 1 , further comprising:
generating the set of recommendations based on availability of items associated with recommendations at a source location associated with the session.
8 . The method of claim 1 , wherein the scoring model is a machine-learning model that was trained by:
accessing a set of training examples including training user data and training order data for a set of training recommendations;
applying the scoring model to the set of training examples to generate a training output corresponding to a set of training scores for the set of training recommendations;
back-propagating one or more error terms obtained from one or more loss functions to update a set of parameters of the scoring model, and one or more of the error terms are based on a difference between a label applied to a test interaction of the set of training examples and the set of training scores; and
stopping the back-propagation after the one or more loss functions satisfy one or more criteria.
9 . The method of claim 1 , further comprising:
generating additional training examples using recommendations that were provided to user devices and conversions associated with the recommendations; and
retraining the scoring model based in part on the additional training examples.
10 . A computer program product comprising a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by a processor of a computer system, cause the computer system to perform steps comprising:
responsive to a page load event that is part of a session between the computer system and a user device of a user, identifying a set of recommendations, wherein the page load event is associated with a target latency period;
retrieving long-term features for each of the set of recommendations, wherein each of the long-term features describes a relationship between the user and a recommendation of the set of recommendations;
generating short-term features for each of the set of recommendations, wherein each of the short-term features describes a relationship between the session and a recommendation of the set of recommendations;
applying the long-term features and the short-term features to a scoring model that outputs scores for each of the set of recommendations;
selecting a set of candidate recommendations from the set of recommendations based in part on the scores;
providing the selected candidate recommendations to a selector, wherein the selector selects one or more of the candidate recommendations; and
providing the selected one or more candidate recommendations to the user device before expiration of the target latency period, wherein providing the selected one or more candidate recommendations to the user device causes the user device to display the one or more candidate recommendations.
11 . The computer program product of claim 10 , wherein applying the long-term features and the short-term features to the scoring model comprises:
identifying a platform context of the user device; and
applying the long-term features, the short-term features, and the platform context to the scoring model that outputs the scores for the set.
12 . The computer program product of claim 10 , wherein applying the long-term features and the short-term features to the scoring model comprises:
identifying a recommendation slot context for an ordering interface of the user device; and
applying the long-term features, the short-term features, and the recommendation slot context to the scoring model that outputs the scores for the set,
wherein the recommendation provided to the user device is presented via the ordering interface.
13 . The computer program product of claim 10 , wherein generating the short-term features for each of the set of recommendations comprises:
using a session model to generate session scores for each of the set of recommendations, wherein the generated session scores are the short-term features.
14 . The computer program product of claim 10 , further comprising encoded instructions that when executed cause the computer system to perform steps comprising:
before the session, generating relevance scores for each of the set using user embeddings and item embeddings, wherein retrieving the long-term features comprises retrieving the generated relevance scores.
15 . The computer program product of claim 10 , further comprising encoded instructions that when executed cause the computer system to perform steps comprising:
before the session, generating conversion probability scores for the set, wherein retrieving the long-term features comprises retrieving the generated conversion probability scores.
16 . The computer program product of claim 10 , further comprising encoded instructions that when executed cause the computer system to perform steps comprising:
generating the set of recommendations based on availability of items associated with recommendations at a source location associated with the session.
17 . The computer program product of claim 10 , wherein the scoring model is a machine-learning model that was trained by:
accessing a set of training examples including training user data and training order data for a set of training recommendations;
applying the scoring model to the set of training examples to generate a training output corresponding to a set of training scores for the set of training recommendations;
back-propagating one or more error terms obtained from one or more loss functions to update a set of parameters of the scoring model, and one or more of the error terms are based on a difference between a label applied to a test interaction of the set of training examples and the set of training scores; and
stopping the back-propagation after the one or more loss functions satisfy one or more criteria.
18 . The computer program product of claim 10 , further comprising encoded instructions that when executed cause the computer system to perform steps comprising:
generating additional training examples using recommendations that were provided to user devices and conversions associated with the recommendations; and
retraining the scoring model based in part on the additional training examples.
19 . A computer system comprising:
a processor; and
a non-transitory computer readable storage medium having instructions encoded thereon that, when executed by the processor, cause the computer system to perform steps comprising:
responsive to a page load event that is part of a session between the computer system and a user device of a user, identifying a set of recommendations, wherein the page load event is associated with a target latency period;
retrieving long-term features for each of the set of recommendations, wherein each of the long-term features describes a relationship between the user and a recommendation of the set of recommendations;
generating short-term features for each of the set of recommendations, wherein each of the short-term features describes a relationship between the session and a recommendation of the set of recommendations;
applying the long-term features and the short-term features to a scoring model that outputs scores for each of the set of recommendations;
selecting a set of candidate recommendations from the set of recommendations based in part on the scores;
providing the selected candidate recommendations to a selector, wherein the selector selects one or more of the candidate recommendations; and
providing the selected one or more candidate recommendations to the user device before expiration of the target latency period, wherein providing the selected one or more candidate recommendations to the user device causes the user device to display the one or more candidate recommendations.
20 . The computer system of claim 19 , wherein applying the long-term features and the short-term features to the scoring model comprises:
identifying a platform context of the user device; and
applying the long-term features, the short-term features, and the platform context to the scoring model that outputs the scores for the set.