Latency and CPU utilization control for content item selection
When an online system receives a request to present content items to a user, a content selection system included in the online system selects content items for presentation to the user. A feedback control mechanism communicates with each computing device of the content selection system to determine the latency period and the CPU utilization of each computing device. The feedback control mechanism also determines a target latency period and a target CPU utilization in which content items are selected. By comparing the latency period of each computing device to the target latency period and the CPU utilization to the target CPU utilization, an amount of information to be evaluated by each computing device is determined based on the comparisons.
1. A method comprising:
receiving a request for one or more content items for a user;
receiving a set of candidate content items for evaluation in a content selection process, the content selection process controlled by a latency period parameter and a CPU utilization parameter;
retrieving a target latency period and a target CPU utilization for the content selection process;
evaluating the set of candidate content items in the content selection process;
determining an actual latency period of the content selection process for the set of candidate content items;
updating the latency period parameter of the content selection process based on a comparison of the target latency period and the actual latency period;
determining an actual CPU utilization of the content selection process for the set of candidate content items;
updating the CPU utilization parameter based on a comparison of the target CPU utilization and the actual CPU utilization; and
evaluating a different set of candidate content items using the updated latency period parameter and the updated CPU utilization parameter in the content selection process.
2. The method of claim 1 , wherein updating the latency period parameter comprises:
determining a number of candidate content items in the set of candidate content items corresponding to the actual latency period;
comparing the actual latency period to the target latency period; and
responsive to determining that the actual latency period is shorter than the target latency period, setting a number of candidate content items in the different set of content items to be greater than the number of candidate content items in the set of content items.
3. The method of claim 1 , wherein the latency period parameter is updated to a maximum latency period value.
4. The method of claim 1 , wherein updating the CPU utilization parameter comprises:
determining a number of candidate content items in the set of candidate content items corresponding to the actual CPU utilization;
comparing the actual CPU utilization to the target CPU utilization; and
responsive to determining that the actual CPU utilization is lower than the target CPU utilization, setting a number of candidate content items in the different set of content items to be greater than the number of candidate content items in the set of content items.
5. The method of claim 1 , wherein updating the CPU utilization parameter comprises:
determining a number of threads used to evaluate the set of candidate content items associated with the actual CPU utilization;
comparing the actual CPU utilization to the target CPU utilization; and
responsive to determining that the actual CPU utilization is lower than the target CPU utilization, setting a number of threads in the evaluation of the different set of candidate content items to be greater than the number of threads used in the evaluation of the set of candidate content items.
6. The method of claim 5 , wherein the set of candidate content items are divided into one or more batches, each batch containing a portion of the set of candidate content items, and wherein each thread of the number of threads is associated with a batch.
7. The method of claim 5 , wherein setting the number of threads in the evaluation of the different set of candidate content items comprises:
determining a processing rate by dividing the number of candidate content items in the set of candidate content items by a timeout value; and
determining the number of threads by dividing the determined processing rate by a theoretical processing rate.
8. The method of claim 7 , further comprising:
comparing the determined number of threads to a minimum number of threads; and
responsive to determining that the number of threads is smaller than the minimum number of threads, setting the number of threads to the minimum number of threads.
9. The method of claim 7 , further comprising:
comparing the determined number of threads to a maximum number of threads; and
responsive to determining that the number of threads is greater than the maximum number of threads, setting the number of threads to the maximum number of threads.
10. The method of claim 1 , wherein the target latency period is a percentile of prior latency periods of a device.
11. The method of claim 1 , wherein the content selection process has multiple stages.
12. The method of claim 1 , wherein the content selection process comprises:
determining a score for each of the candidate content items by applying a trained machine learning model to a set of characteristics of the user and a set of characteristics of the candidate content item, the machine learning model predicting a likelihood the user's interaction with the candidate content item; and
selecting one or more of the candidate content items based on the scores.
13. A computer program product comprising a non-transitory computer-readable storage medium having instructions encoded thereon that, when executed by a processor, cause the processor to:
receive a request for one or more content items for a user;
receive a set of candidate content items for evaluation in a content selection process, the content selection process controlled by a latency period parameter and a CPU utilization parameter;
retrieve a target latency period and a target CPU utilization for the content selection process;
evaluate the set of candidate content items in the content selection process;
determine an actual latency period of the content selection process for the set of candidate content items;
update the latency period parameter of the content selection process based on a comparison of the target latency period and the actual latency period;
determine an actual CPU utilization of the content selection process for the set of candidate content items;
update the CPU utilization parameter based on a comparison of the target CPU utilization and the actual CPU utilization; and
evaluate a different set of candidate content items using the updated latency period parameter and the updated CPU utilization parameter in the content selection process.
14. The computer program product of claim 13 , wherein updating the latency period comprises:
determining a number of candidate content items in the set of candidate content items corresponding to the actual latency period;
comparing the actual latency period to the target latency period; and
responsive to determining that the actual latency period is shorter than the target latency period, setting a number of candidate content items in the different set of content items to be greater than the number of candidate content items in the set of content items.
15. The computer program product of claim 13 , wherein the latency period parameter is updated to a maximum latency period value.
16. The computer program product of claim 13 , wherein updating the CPU utilization parameter comprises:
determining a number of candidate content items in the set of candidate content items corresponding to the actual CPU utilization;
comparing the actual CPU utilization to the target CPU utilization; and
responsive to determining that the actual CPU utilization is lower than the target CPU utilization, setting a number of candidate content items in the different set of content items to be greater than the number of candidate content items in the set of content items.
17. The computer program product of claim 13 , wherein updating the CPU utilization parameter comprises:
determining a number of threads used to evaluate the set of candidate content items associated with the actual CPU utilization;
comparing the actual CPU utilization to the target CPU utilization; and
responsive to determining that the actual CPU utilization is lower than the target CPU utilization, setting a number of threads in the evaluation of the different set of candidate content items to be greater than the number of threads used in the evaluation of the set of candidate content items.
18. The computer program product of claim 17 , wherein setting the number of threads in the evaluation of the different set of content items further comprises:
determining a processing rate by dividing the number of candidate content items in the set of candidate content items by a timeout value; and
determining the number of threads by dividing the determined processing rate by a theoretical processing rate.
19. The computer program product of claim 13 , wherein the target latency period is a percentile of prior latency periods of a device.
20. The computer program product of claim 13 , wherein the content selection process comprises:
determining a score for each of the candidate content items by applying a trained machine learning model to a set of characteristics of the user and a set of characteristics of the candidate content item, the machine learning model predicting a likelihood the user's interaction with the candidate content item; and
selecting one or more of the candidate content items based on the scores.