Machine learning in resource-constrained environments
In one embodiment, a method includes receiving a request to determine whether to perform an action, wherein the action is based on one or more feature values, generating a prediction of whether to perform the action, wherein the prediction is generated using a machine-learning model that is trained based on the feature values, a heuristic value based on the feature values, and one or more feedback scores based on corresponding past predictions generated by the machine-learning model, where the heuristic value indicates whether to perform the action based on one or more predetermined conditions that are based on the feature values, performing the action when the prediction indicates that the action is to be performed, receiving a feedback score that indicates a level of effectiveness of the prediction, and updating the machine-learning model based on the feedback score, the feature values, and the heuristic value.
1. A method comprising:
by a computing device, receiving a request to determine whether to perform an action for processing a data item, wherein the action is based on one or more feature values associated with the data item;
by the computing device, generating a prediction of whether to perform the action, wherein the prediction is generated using a machine-learning model that is trained based on the feature values, a heuristic value based on the feature values, and one or more feedback scores based on corresponding past predictions generated by the machine-learning model,
wherein the heuristic value indicates whether to perform the action based on one or more predetermined conditions that are based on the feature values;
by the computing device, determining, based on a processor utilization level and a number of available processors, whether resources on the computing device are available for updating the machine-learning model; and
by the computing device based on the determination,
updating the machine-learning model online while the computing device performs other operations when resources on the computing device are available, wherein the resources on the computing device being available indicates the processor utilization level is above a threshold level and the number of available processors is greater than a threshold number, and
updating the machine-learning model offline based on backpropagation using low-overhead upgrade operations when resources on the computing device are not available, wherein the resources on the computing device being not available indicates the processor utilization level is not above the threshold level and the number of available processors is not greater than the threshold number.
2. The method of claim 1 , further comprising receiving a feedback score that indicates a level of effectiveness of the prediction, wherein updating the machine-learning model online comprises:
by the computing device, generating one or more online updates for the machine-learning model based on the feedback score, the feature values, and the heuristic value; and
by the computing device, updating the machine-learning model in accordance with the online updates.
3. The method of claim 1 , wherein determining whether resources on the computing device are available for updating the machine-learning model comprises determining whether a resource availability condition is satisfied, and wherein the resource availability condition is based on one or more of: a minimum time interval between updates to the machine-learning model, the processor utilization level, or the number of available processors of the computing device.
4. The method of claim 1 , wherein the updating the machine-learning model offline comprises:
by the computing device, sending the feedback score, the feature values, and the heuristic value to a remote system, wherein the remote system generates one or more offline updates to the machine-learning model based on the feedback score, the feature values, and the heuristic.
5. The method of claim 4 , wherein the sending occurs when a resource availability condition is not satisfied.
6. The method of claim 4 , further comprising:
by the computing device, receiving the offline updates to the machine-learning model from the remote system; and
by the computing device, updating the machine-learning model based on the offline updates.
7. The method of claim 1 , wherein the machine-learning model comprises a reinforcement model.
8. The method of claim 1 , wherein the action comprises admitting an object to a cache, and the feature values comprise one or more attributes of the object.
9. The method of claim 8 , wherein the feedback score is based on a number of requests received for the object since the object was admitted to the cache.
10. The method of claim 1 , wherein the action comprises invalidating a cache entry, and the feature values comprise one or more attributes of the cache entry.
11. The method of claim 10 , further comprising receiving a feedback score that indicates a level of effectiveness of the prediction, wherein the feedback score is based on a number of requests received for the cache entry since the cache entry was invalidated.
12. The method of claim 1 , wherein updating the machine-learning model online comprises:
updating the machine-learning model on the computing device while the computing device is operational.
13. One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
by a computing device, receive a request to determine whether to perform an action for processing a data item, wherein the action is based on one or more feature values associated with the data item;
by the computing device, generate a prediction of whether to perform the action, wherein the prediction is generated using a machine-learning model that is trained based on the feature values, a heuristic value based on the feature values, and one or more feedback scores based on corresponding past predictions generated by the machine-learning model,
wherein the heuristic value indicates whether to perform the action based on one or more predetermined conditions that are based on the feature values;
by the computing device, determine, based on a processor utilization level and a number of available processors, whether resources on the computing device are available for updating the machine-learning model; and
by the computing device based on the determination,
update the machine-learning model online while the computing device performs other operations when resources on the computing device are available, wherein the resources on the computing device being available indicates the processor utilization level is above a threshold level and the number of available processors is greater than a threshold number, and
update the machine-learning model offline based on backpropagation using low-overhead upgrade operations when resources on the computing device are not available, wherein the resources on the computing device being not available indicates the processor utilization level is not above the threshold level and the number of available processors is not greater than the threshold number.
14. The media of claim 13 , wherein the software is further operable when executed to receive a feedback score that indicates a level of effectiveness of the prediction, wherein updating the machine-learning model online comprises:
generating one or more online updates for the machine-learning model based on the feedback score, the feature values, and the heuristic value; and
updating the machine-learning model in accordance with the online updates.
15. The media of claim 13 , wherein determining whether resources on the computing device are available for updating the machine-learning model comprises determining whether a resource availability condition is satisfied, and wherein the resource availability condition is based on one or more of: a minimum time interval between updates to the machine-learning model, the processor utilization level, or the number of available processors of the computing device.
16. The media of claim 13 , wherein to update the machine-learning model offline the software is further operable when executed to:
send the feedback score, the feature values, and the heuristic value to a remote system, wherein the remote system generates one or more offline updates to the machine-learning model based on the feedback score, the feature values, and the heuristic.
17. The media of claim 16 , wherein the sending occurs when a resource availability condition is not satisfied.
18. A system comprising:
one or more processors; and
one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to:
receive a request to determine whether to perform an action for processing a data item, wherein the action is based on one or more feature values associated with the data item;
generate a prediction of whether to perform the action, wherein the prediction is generated using a machine-learning model that is trained based on the feature values, a heuristic value based on the feature values, and one or more feedback scores based on corresponding past predictions generated by the machine-learning model,
wherein the heuristic value indicates whether to perform the action based on one or more predetermined conditions that are based on the feature values;
determine, based on a processor utilization level and a number of available processors, whether resources on the computing device are available for updating the machine-learning model; and
based on the determination,
update the machine-learning model online while the computing device performs other operations when resources on the computing device are available, wherein the resources on the computing device being available indicates the processor utilization level is above a threshold level and the number of available processors is greater than a threshold number, and
update the machine-learning model offline based on backpropagation using low-overhead upgrade operations when resources on the computing device are not available, wherein the resources on the computing device being not available indicates the processor utilization level is not above the threshold level and the number of available processors is not greater than the threshold number.
19. The system of claim 18 , wherein the processors are further operable when executed to receive a feedback score that indicates a level of effectiveness of the prediction, wherein updating the machine-learning model online comprises:
generating one or more online updates for the machine-learning model based on the feedback score, the feature values, and the heuristic value; and
updating the machine-learning model in accordance with the online updates.
20. The system of claim 18 , wherein determining whether resources on the one or more of the processors are available for updating the machine-learning model comprises determining whether a resource availability condition is satisfied, and wherein the resource availability condition is based on one or more of: a minimum time interval between updates to the machine-learning model, the processor utilization level, or the number of available processors of the computing device.
21. The system of claim 18 , wherein to update the machine-learning model offline the processors are further operable when executing the instructions to:
send the feedback score, the feature values, and the heuristic value to a remote system, wherein the remote system generates one or more offline updates to the machine-learning model based on the feedback score, the feature values, and the heuristic.