Methods and systems for managing an accelerator's primary storage unit
The present disclosure provides methods, systems, and non-transitory computer readable media for managing a primary storage unit of an accelerator. The methods include assessing activity of the accelerator; assigning, based on the assessed activity of the accelerator, a lease to a group of one or more pages of data on the primary storage unit, wherein the assigned lease indicates a lease duration; and marking, in response to the expiration of the lease duration indicated by the lease, the group of one or more pages of data as an eviction candidate.
1. A method for managing a primary storage unit of an accelerator, the method comprising:
assessing activity of the accelerator;
assigning, based on the assessed activity of the accelerator, a lease having a predetermined span of time as a lease duration to a group of one or more pages of data on the primary storage unit, wherein the predetermined span of time is determined, before assigning the lease, based on assessment of activity of the accelerator including usage of primary storage unit and of kernels being executed by the accelerator; and
marking, in response to expiration of the lease duration indicated by the lease, the group of one or more pages of data as an eviction candidate.
2. The method of claim 1 , further comprising:
reassessing the activity of the accelerator; and
updating, based on the reassessed activity of the accelerator, the lease duration indicated by the lease assigned to the group of one or more pages.
3. The method of claim 1 , wherein reassessing the activity of the accelerator and updating the lease duration lease occurs when:
one of the pages in the group of one or more pages is referenced,
a page not in the group of one or more pages is referenced,
a threshold amount of time has passed,
a threshold amount of time has passed without the lease being reevaluated, or
an update epoch is reached.
4. The method of claim 1 , further comprising:
determining, based on the assessed activity of accelerator, a page prefetch amount; and
retrieving the group of one or more pages of data from a memory system connected to the accelerator, wherein:
the group of one or more pages contains one or more referenced pages and zero or more prefetched pages, and
the amount of prefetched pages retrieved is based on the page prefetch amount.
5. The method of claim 4 , further comprising:
monitoring memory activity referencing any groups of one or more pages on the primary storage unit; and
based on the monitored memory activity and the assessed activity of the accelerator, adjusting a fetching strategy that is used to determine the page prefetch amount.
6. The method of claim 4 , wherein the retrieval of the group of one or more pages of data was triggered by a kernel being executed on the accelerator attempting to access the one or more referenced pages.
7. The method of claim 4 , wherein:
the memory system is a primary storage unit of a host system, and
the accelerator and host system use a unified virtual memory system.
8. The method of claim 1 , further comprising removing, based on the activity of the accelerator, one or more groups of one or more pages marked as eviction candidates from the accelerator's primary storage unit.
9. The method of claim 8 , further comprising:
monitoring memory activity referencing any groups of one or more pages removed from the accelerator's primary storage unit; and
based on the monitored memory activity, adjusting an eviction strategy that is used for removing the one or more groups of one or more pages marked as eviction candidates.
10. The method of claim 1 , wherein the activity of the accelerator comprises:
an amount of the accelerator's primary storage unit being used by a triggering kernel being executed on the accelerator, wherein the triggering kernel triggered retrieval of the one or more pages of data,
an amount of the accelerator's primary storage unit being used by any kernel being executed on the accelerator that is not the triggering kernel,
an amount of the accelerator's primary storage unit not being used by any kernel being executed on the accelerator,
an amount of the accelerator's primary storage unit not being used by any kernel being executed on the accelerator that is not the triggering kernel,
lease durations for any of the accelerator's primary storage unit being used by the triggering kernel,
lease durations for any of the accelerator's primary storage unit being used by any kernel being executed on the accelerator that is not the triggering kernel,
memory access patterns amount of the triggering kernel, or
memory access patterns amount of any kernel being executed on the accelerator that is not the triggering kernel.
11. A system for managing a primary storage unit of an accelerator, the system comprising:
an assessment unit configured to assess activity of the accelerator;
a leasing unit configured to assign, based on the assessed activity of the accelerator, a lease having a predetermined span of time as a lease duration to a group of one or more pages of data on the primary storage unit, wherein the predetermined span of time is determined, before assigning the lease, based on assessment of the activity of the accelerator including usage of primary storage unit and of kernels being executed by the accelerator; and
an eviction unit configured to mark, in response to expiration of the lease duration indicated by the lease, the group of one or more pages of data as an eviction candidate.
12. The system of claim 11 , wherein:
the assessment unit is further configured to reassess the activity of the accelerator; and
the leasing unit is further configured to update, based on the reassessed activity of the accelerator, the lease duration indicated by the lease assigned to the group of one or more pages.
13. The system of claim 11 , further comprising:
a prefetch determination unit configured to determine, based on the assessed activity of accelerator, a page prefetch amount; and
a fetching unit configured to fetch the group of one or more pages of data from a memory system connected to the accelerator, wherein:
the group of one or more pages contains one or more referenced pages and zero or more prefetched pages, and
the amount of prefetched pages retrieved is based on the page prefetch amount.
14. The system of claim 11 , wherein the eviction unit is further configured to remove, based on the activity of the accelerator, one or more groups of one or more pages marked as eviction candidates from the accelerator's primary storage unit.
15. The system of claim 14 , further comprising a memory monitoring unit configured to monitor memory activity referencing any groups of one or more pages removed from the accelerator's primary storage unit; and
wherein the eviction unit is further configured to adjust, based on the monitored memory activity, an eviction strategy that is used for removing the one or more groups of one or more pages marked as eviction candidates.
16. A non-transitory computer readable medium that stores a set of instructions that is executable by at least one processor of a computer system to cause the computer system to perform a method for managing a primary storage unit of an accelerator, the method comprising:
assessing activity of the accelerator;
assigning, based on the assessed activity of the accelerator, a lease to a having a predetermined span of time as a lease duration to a group of one or more pages of data on the primary storage unit, wherein the predetermined span of time is determined, before assigning the lease, based on assessment of activity of the accelerator including usage of primary storage unit and of kernels being executed by the accelerator; and
marking, in response to expiration of the lease duration indicated by the lease, the group of one or more pages of data as an eviction candidate.
17. The non-transitory computer readable medium of claim 16 , wherein the set of instructions is executable by the at least one processor of the computer system to cause the computer system to further perform:
reassessing the activity of the accelerator; and
updating, based on the reassessed activity of the accelerator, the lease duration indicated by the lease assigned to the group of one or more pages.
18. The non-transitory computer readable medium of claim 16 , wherein the set of instructions is executable by the at least one processor of the computer system to cause the computer system to further perform:
determining, based on the assessed activity of accelerator, a page prefetch amount; and
retrieving the group of one or more pages of data from a memory system connected to the accelerator, wherein:
the group of one or more pages contains one or more referenced pages and zero or more prefetched pages, and
the amount of prefetched pages retrieved is based on the page prefetch amount.
19. The non-transitory computer readable medium of claim 16 , wherein the set of instructions is executable by the at least one processor of the computer system to cause the computer system to further perform removing, based on the activity of the accelerator, one or more groups of one or more pages marked as eviction candidates from the accelerator's primary storage unit.
20. The non-transitory computer readable medium of claim 19 , wherein the set of instructions is executable by the at least one processor of the computer system to cause the computer system to further perform:
monitoring memory activity referencing any groups of one or more pages removed from the accelerator's primary storage unit; and
based on the monitored memory activity, adjusting an eviction strategy that is used for removing the one or more groups of one or more pages marked as eviction candidates.