Systems and methods for delayed allocation in cluster storage
The disclosed computer-implemented method for delayed allocation in cluster storage may include (i) delegating, to a node attached to a storage cluster including one or more storage devices that include a group of allocation units, a subset of allocation units on the storage cluster to be held as a delayed allocation pool, (ii) receiving, from the node, a request to allocate allocation units on the storage cluster, (iii) deducting the number of allocation units from available allocation units in the delayed allocation pool, (iv) satisfying the allocation request by allocating allocation units not in the delayed allocation pool before allocating allocation units included in the delayed allocation pool, and (v) recalculating, based on the number of allocation units in the delayed allocation pool used to satisfy the allocation request, the number of available allocation units in the delayed allocation pool. Various other methods, systems, and computer-readable media are also disclosed.
1. A computer-implemented method for delayed allocation in cluster storage, at least a portion of the method being performed by a computing device comprising at least one processor, the method comprising:
delegating, to a node attached to a storage cluster comprising at least one storage device that comprises a plurality of allocation units, a subset of the allocation units on the storage cluster to be held as a delayed allocation pool;
receiving, from the node, a request to allocate a number of allocation units on the storage cluster;
deducting the number of allocation units from a number of available allocation units in the delayed allocation pool;
satisfying the allocation request by allocating allocation units not included in the delayed allocation pool before allocating allocation units included in the delayed allocation pool;
recalculating, based on the number of allocation units in the delayed allocation pool used to satisfy the allocation request, the number of available allocation units in the delayed allocation pool;
maintaining, for the node, a measurement of a rate at which the node requests allocation units on the storage cluster;
comparing the allocation request rate for the node to the number of allocation units held in the delayed allocation pool; and
adjusting the number of allocation units in the delayed allocation pool by delegating additional allocation units to the delayed allocation pool or relinquishing a delegation of allocation units to the delayed allocation pool based at least in part on the comparison.
2. The computer-implemented method of claim 1 , further comprising:
determining that the number of available allocation units in the delayed allocation pool is below a threshold; and
in response to determining that the number of available allocation units in the delayed allocation pool is below the threshold, replenishing the delayed allocation pool by delegating additional allocation units on the storage cluster to the delayed allocation pool.
3. The computer-implemented method of claim 1 , further comprising:
determining that the number of available allocation units in the delayed allocation pool is below a threshold;
determining that the number of available allocation units on the storage cluster is below an available unit threshold; and
in response to determining that the number of available allocation units on the storage cluster is below the available unit threshold:
identifying at least one additional node attached to the storage cluster with allocation units on the storage cluster delegated to the additional node;
revoking the delegation of a subset of the allocation units delegated to the additional node; and
delegating the subset of allocation units to the delayed allocation pool.
4. The computer-implemented method of claim 1 , further comprising:
determining that the number of available allocation units in the delayed allocation pool is below a threshold;
determining that the number of available allocation units on the storage cluster is below an available unit threshold; and
in response to determining that the number of available allocation units on the storage cluster is below the available unit threshold, reporting that additional allocation units cannot be delegated to the delayed allocation pool.
5. The computer-implemented method of claim 1 , wherein the allocation units are delegated to be held as the delayed allocation pool in response to at least one of:
a file system mount event;
a node join event; and
a first delayed allocation event.
6. The computer-implemented method of claim 1 , wherein the number of allocation units in the delayed allocation pool is determined at least in part by the number of available allocation units in the storage cluster.
7. The computer-implemented method of claim 1 ,
wherein adjusting the number of allocation units in the delayed allocation pool by delegating additional allocation units to the delayed allocation pool or relinquishing a delegation of allocation units to the delayed allocation pool is further based at least in part on at least one of:
a rate of allocation of allocation units in the delayed allocation pool;
a ratio of the node's allocation request rate to an allocation request rate for at least one additional node attached to the storage cluster;
a ratio of the node's allocation request rate to a number of allocation units allocated to the node, but not in the delayed allocation pool; and
a ratio of the node's allocation request rate to a number of available allocation units on the storage cluster.
8. The computer-implemented method of claim 1 , wherein the allocation request is satisfied by allocating allocation units delegated to the node, but not included in the delayed allocation pool.
9. The computer-implemented method of claim 1 , wherein the delegation of allocation units to the delayed allocation pool is made from a process executing at a lower priority than a priority of a process servicing allocation requests from the node.
10. A system for delayed allocation in cluster storage, the system comprising:
a delegation module, stored in memory, that delegates, to a node attached to a storage cluster comprising at least one storage device that comprises a plurality of allocation units, a subset of the allocation units on the storage cluster to be held as a delayed allocation pool;
a communication module, stored in memory, that receives, from the node, a request to allocate a number of allocation units on the storage cluster;
an adjustment module, stored in memory, that deducts the number of allocation units from a number of available allocation units in the delayed allocation pool, wherein the adjustment module:
maintains, for the node, a measurement of a rate at which the node requests allocation units on the storage cluster;
compares the allocation request rate for the node to the number of allocation units held in the delayed allocation pool; and
adjusts the number of allocation units in the delayed allocation pool by delegating additional allocation units to the delayed allocation pool or relinquishing a delegation of allocation units to the delayed allocation pool based at least in part on the comparison;
an allocation module, stored in memory, that satisfies the allocation request by allocating allocation units not included in the delayed allocation pool before allocating allocation units included in the delayed allocation pool, wherein the adjustment module recalculates, based on the number of allocation units in the delayed allocation pool used to satisfy the allocation request, the number of available allocation units in the delayed allocation pool; and
at least one physical processor configured to execute the delegation module, the communication module, the adjustment module, and the allocation module.
11. The system of claim 10 , wherein the adjustment module:
determines that the number of available allocation units in the delayed allocation pool is below a threshold; and
in response to determining that the number of available allocation units in the delayed allocation pool is below the threshold, replenishes the delayed allocation pool by delegating additional allocation units on the storage cluster to the delayed allocation pool.
12. The system of claim 10 , wherein the adjustment module:
determines that the number of available allocation units in the delayed allocation pool is below a threshold;
determines that the number of available allocation units on the storage cluster is below an available unit threshold; and
in response to determining that the number of available allocation units on the storage cluster is below the available unit threshold:
identifies at least one additional node attached to the storage cluster with allocation units on the storage cluster delegated to the additional node;
revokes the delegation of a subset of the allocation units delegated to the additional node; and
delegates the subset of allocation units to the delayed allocation pool.
13. The system of claim 10 , wherein the adjustment module:
determines that the number of available allocation units in the delayed allocation pool is below a threshold;
determines that the number of available allocation units on the storage cluster is below an available unit threshold; and
in response to determining that the number of available allocation units on the storage cluster is below the available unit threshold, reports that additional allocation units cannot be delegated to the delayed allocation pool.
14. The system of claim 10 , wherein the delegation module delegates the allocation units to be held as the delayed allocation pool in response to at least one of:
a file system mount event;
a node join event; and
a first delayed allocation event.
15. The system of claim 10 , wherein the number of allocation units the delegation module delegates to the delayed allocation pool is determined at least in part by the number of available allocation units in the storage cluster.
16. The system of claim 10 , wherein the adjustment module:
adjusts the number of allocation units in the delayed allocation pool by delegating additional allocation units to the delayed allocation pool or relinquishing a delegation of allocation units to the delayed allocation pool further based at least in part on at least one of:
a rate of allocation of allocation units in the delayed allocation pool;
a ratio of the node's allocation request rate to an allocation request rate for at least one additional node attached to the storage cluster;
a ratio of the node's allocation request rate to a number of allocation units allocated to the node, but not in the delayed allocation pool; and
a ratio of the node's allocation request rate to a number of available allocation units on the storage cluster.
17. The system of claim 10 , wherein the allocation module satisfies the allocation request by allocating allocation units delegated to the node, but not included in the delayed allocation pool.
18. The system of claim 10 , wherein the adjustment module delegates allocation units to the delayed allocation pool from a process executing at a lower priority than a priority of a process servicing allocation requests from the node.
19. A non-transitory computer-readable medium comprising one or more computer-readable instructions that, when executed by at least one processor of a computing device, cause the computing device to:
delegate, to a node attached to a storage cluster comprising at least one storage device that comprises a plurality of allocation units, a subset of the allocation units on the storage cluster to be held as a delayed allocation pool;
receive, from the node, a request to allocate a number of allocation units on the storage cluster;
deduct the number of allocation units from a number of available allocation units in the delayed allocation pool;
satisfy the allocation request by allocating allocation units not included in the delayed allocation pool before allocating allocation units included in the delayed allocation pool;
recalculate, based on the number of allocation units in the delayed allocation pool used to satisfy the allocation request, the number of available allocation units in the delayed allocation pool;
maintain, for the node, a measurement of a rate at which the node requests allocation units on the storage cluster;
compare the allocation request rate for the node to the number of allocation units held in the delayed allocation pool; and
adjust the number of allocation units in the delayed allocation pool by delegating additional allocation units to the delayed allocation pool or relinquishing a delegation of allocation units to the delayed allocation pool based at least in part on the comparison.
20. The non-transitory computer-readable medium of claim 19 , wherein the one or more computer-readable instructions cause the computing device to:
determine that the number of available allocation units in the delayed allocation pool is below a threshold; and
in response to determining that the number of available allocation units in the delayed allocation pool is below the threshold, replenish the delayed allocation pool by delegating additional allocation units on the storage cluster to the delayed allocation pool.