Controlling cache size and priority using machine learning techniques
Techniques are disclosed relating to controlling cache size and priority of data stored in the cache using machine learning techniques. A software cache may store data for a plurality of different user accounts using one or more hardware storage elements. In some embodiments, a machine learning module generates, based on access patterns to the software cache, a control value that specifies a size of the cache and generates time-to-live values for entries in the cache. In some embodiments, the system evicts data based on the time-to-live values. The disclosed techniques may reduce cache access times and/or improve cache hit rate.
1. An system for controlling cache size, comprising:
a non-transitory memory; and
one or more hardware processors coupled to the non-transitory memory and configured to read instructions from the non-transitory memory to cause the system to perform operations comprising:
reading user account information from a software cache;
updating, based on the reading of the user account information, access history of the software cache associated with the user account information;
determining a total read score for the user account information and a read time score for the user account information;
generating a relevance score for the user account information based on the total read score and the read time score;
determining an amount of space available in the software cache; and
updating time-to-live values for the user account information and a size of the software cache based on the relevance score and the amount of space available in the software cache.
2. The system of claim 1 , wherein the total read score is determined based on at least one of a total number of times a cache entry is accessed in the software cache or a total amount of data being accessed.
3. The system of claim 1 , wherein the read time score is determined based on an amount of time between a previous access and a current access of a cache entry from the software cache.
4. The system of claim 3 , wherein the read time score decreases as the amount of time between the previous access and the current access of the cache entry increases.
5. The system of claim 1 , wherein the read time score is determined based on at least one of a number of accesses within a set of intervals or a detected pattern of accesses.
6. The system of claim 1 , wherein the relevance score is generated by a machine learning module that adjusts the relevance score based on the total read score and the read time scores during a training.
7. The system of claim 1 , wherein the amount of space available in the software cache is determined based on a difference between a space allocated for a current software cache size and a space used of the current software cache size.
8. A method for controlling cache size, comprising:
reading user account information from a software cache;
updating, based on the reading of the user account information, access history of the software cache associated with the user account information;
determining a total read score for the user account information and a read time score for the user account information;
generating a relevance score for the user account information based on the total read score and the read time score;
determining an amount of space available in the software cache; and
updating time-to-live values for the user account information and a size of the software cache based on the relevance score and the amount of space available in the software cache.
9. The method of claim 8 , wherein the total read score is determined based on at least one of a total number of times a cache entry is accessed in the software cache or a total amount of data being accessed.
10. The method of claim 8 , wherein the read time score is determined based on an amount of time between a previous access and a current access of a cache entry from the software cache.
11. The method of claim 10 , wherein the read time score decreases as the amount of time between the previous access and the current access of the cache entry increases.
12. The method of claim 8 , wherein the read time score is determined based on at least one of a number of accesses within a set of intervals or a detected pattern of accesses.
13. The method of claim 8 , wherein the relevance score is generated by a machine learning module that adjusts the relevance score based on the total read score and the read time scores during a training.
14. The method of claim 8 , wherein the amount of space available in the software cache is determined based on a difference between a space allocated for a current software cache size and a space used of the current software cache size.
15. A non-transitory computer-readable medium having instructions stored thereon that are executable by a computing system to perform operations comprising:
reading user account information from a software cache;
updating, based on the reading of the user account information, access history of the software cache associated with the user account information;
determining a total read score for the user account information and a read time score for the user account information;
generating a relevance score for the user account information based on the total read score and the read time score;
determining an amount of space available in the software cache; and
updating time-to-live values for the user account information and a size of the software cache based on the relevance score and the amount of space available in the software cache.
16. The non-transitory computer-readable medium of claim 15 , wherein the total read score is determined based on at least one of a total number of times a cache entry is accessed in the software cache or a total amount of data being accessed.
17. The non-transitory computer-readable medium of claim 15 , wherein the read time score is determined based on an amount of time between a previous access and a current access of a cache entry from the software cache, and wherein the read time score decreases as the amount of time between the previous access and the current access of the cache entry increases.
18. The non-transitory computer-readable medium of claim 15 , wherein the read time score is determined based on at least one of a number of accesses within a set of intervals or a detected pattern of accesses.
19. The non-transitory computer-readable medium of claim 15 , wherein the relevance score is generated by a machine learning module that adjusts the relevance score based on the total read score and the read time scores during a training.
20. The non-transitory computer-readable medium of claim 15 , wherein the amount of space available in the software cache is determined based on a difference between a space allocated for a current software cache size and a space used of the current software cache size.