Adaptive cache commit delay for write aggregation

1. A system, comprising:

a data cache configured to aggregate a plurality of file data requests in at least one buffer segment in the data cache; and

a cache manager configured to:

determine a percentage of writes to a persistent storage medium that meet a buffer threshold, wherein:

the buffer threshold is a predetermined amount of buffer space occupied by file data in the at least one buffer segment for a write operation to the persistent storage medium;

the at least one buffer segment has a serialized set of buffer positions;

the file data in the at least one buffer segment includes a plurality of file block sizes from the plurality of file data requests; and

each file block of the file data in the at least one buffer segment is assigned to at least one buffer position using a caching algorithm;

determine, based on the percentage of writes, a delay threshold for the at least one buffer segment, wherein the delay threshold is a time value;

monitor a commit time value for the at least one buffer segment after a first file data element is stored in the at least one buffer segment;

determine, based on the commit time value satisfying the delay threshold, a time-based data commit trigger for the at least one buffer segment, wherein the time-based data commit trigger is set independently of a current amount of buffer space occupied by aggregate data elements in the at least one buffer segment; and

move, responsive to a first data commit trigger, aggregate data elements from the at least one buffer segment to the persistent storage medium.

2. The system of claim 1 , further comprising:

a file server configured to manage the plurality of file data requests, wherein the plurality of file data requests are received from at least one client system using a file system protocol; and

a persistent storage interface configured to store, responsive to the cache manager, aggregate data elements to the persistent storage medium,

wherein:

the aggregate data elements are from the at least one buffer segment;

the data cache includes a plurality of buffer segments that include the at least one buffer segment;

each buffer segment from the plurality of buffer segments has a buffer size corresponding to a predetermined persistent data element size; and

the cache manager is further configured to:

aggregate file data requests received by the file server in the plurality of buffer segments; and

determine, responsive to aggregate data elements in a buffer segment exceeding the buffer threshold, a complete data commit trigger for the buffer segment, wherein the first data commit trigger is a first in time of the time-based data commit trigger and the complete data commit trigger.

3. The system of claim 1 , wherein the cache manager is further configured to:

determine total writes to the persistent storage medium;

determine, among the total writes to the persistent storage medium, writes meeting the buffer threshold;

determine the percentage of writes is less than a usage target for the data cache; and

apply, responsive to determining the percentage of writes is less than the usage target, an adaptation algorithm to increase the time value for the delay threshold for the at least one buffer segment.

4. The system of claim 3 , wherein the buffer threshold is based on a predetermined persistent data element size for the persistent storage medium.

5. The system of claim 1 , wherein determining the delay threshold further includes:

determining a default delay threshold;

determining that the percentage of writes is less than a usage target for the data cache; and

applying, responsive to determining that the percentage of writes is less than the usage target, an adaptation algorithm to the default delay threshold to increase the time value for the delay threshold.

6. The system of claim 5 , wherein increasing the time value for the delay threshold further includes:

iteratively applying, responsive to determining that the percentage of writes is less than the usage target, the adaptation algorithm to a prior delay threshold to determine the delay threshold.

7. The system of claim 6 , wherein the adaptation algorithm iteratively increments the delay threshold along a curve indexed by the prior delay threshold until a maximum delay threshold is reached.

8. The system of claim 1 , wherein:

the data cache includes a plurality of buffer segments, including the at least one buffer segment;

the plurality of file data requests correspond to a plurality of logical data groups;

each buffer segment from the plurality of buffer segments aggregates file data requests corresponding to one logical data group of the plurality of logical data groups; and

the cache manager is further configured to:

monitor at least one usage value for each logical data group of the plurality of logical data groups;

aggregate file data requests received by a file server in the plurality of buffer segments; and

determine, based on the at least one usage value for each logical data group of the plurality of logical data groups, a group delay threshold for each logical data group of the plurality of logical data groups, wherein determining the delay threshold for the at least one buffer segment includes selecting, based on the one logical data group corresponding to aggregate data elements in the at least one buffer segment, the delay threshold from a corresponding group delay threshold.

9. A computer-implemented method, comprising:

aggregating a plurality of data requests in at least one buffer segment in a data cache;

determining a percentage of writes to a persistent storage node that meet a buffer threshold, wherein:

the buffer threshold is a predetermined amount of buffer space occupied by file data in the at least one buffer segment for a write operation to the persistent storage node;

the at least one buffer segment has a serialized set of buffer positions;

the file data in the at least one buffer segment includes a plurality of file block sizes from the plurality of data requests; and

each file block of the file data in the at least one buffer segment is assigned to at least one buffer position using a caching algorithm;

determining, based on the percentage of writes, a delay threshold for the at least one buffer segment, wherein the delay threshold is a time value;

monitoring a commit time value for the at least one buffer segment after a first data element is stored in the at least one buffer segment;

determining, based on the commit time value satisfying the delay threshold, a time-based data commit trigger for the at least one buffer segment, wherein the time-based data commit trigger is set independently of a current amount of buffer space occupied by aggregate data elements in the at least one buffer segment; and

moving, responsive to a first data commit trigger, aggregate data elements from the at least one buffer segment to the persistent storage node.

10. The computer-implemented method of claim 9 , further comprising:

managing the plurality of data requests, wherein:

the plurality of data requests are received from at least one client system using a file system protocol;

the persistent storage node has a predetermined persistent data element size;

the data cache includes a plurality of buffer segments that include the at least one buffer segment; and

each buffer segment from the plurality of buffer segments has a buffer size corresponding to the predetermined persistent data element size;

aggregating data requests in the plurality of buffer segments; and

determining, responsive to aggregate data elements in a buffer segment exceeding the buffer threshold, a complete data commit trigger for the buffer segment, wherein the first data commit trigger is a first in time of the time-based data commit trigger and the complete data commit trigger.

11. The computer-implemented method of claim 9 , further comprising:

determining total writes to the persistent storage node;

determining, among the total writes to the persistent storage node, writes meeting the buffer threshold;

determining the percentage of writes is less than a usage target for the data cache; and

applying, responsive to determining the percentage of writes is less than the usage target, an adaptation algorithm to increase the time value for the delay threshold for the at least one buffer segment.

12. The computer-implemented method of claim 11 , wherein the buffer threshold is based on a predetermined persistent data element size for the persistent storage node.

13. The computer-implemented method of claim 9 , further comprising:

determining a default delay threshold;

determining that the percentage of writes is less than a usage target for the data cache; and

applying, responsive to determining that the percentage of writes is less than the usage target, an adaptation algorithm to the default delay threshold to increase the time value for the delay threshold.

14. The computer-implemented method of claim 13 , further comprising:

iteratively applying, responsive to determining that the percentage of writes is less than the usage target, the adaptation algorithm to a prior delay threshold to determine the delay threshold.

15. The computer-implemented method of claim 14 , wherein the adaptation algorithm iteratively increments the delay threshold along a curve indexed by the prior delay threshold until a maximum delay threshold is reached.

16. The computer-implemented method of claim 9 , wherein:

the data cache includes a plurality of buffer segments, including the at least one buffer segment;

the plurality of data requests correspond to a plurality of logical data groups;

each buffer segment from the plurality of buffer segments aggregates file data requests corresponding to one logical data group of the plurality of logical data groups; and

further comprising:

monitoring at least one usage value for each logical data group of the plurality of logical data groups;

aggregating data requests in the plurality of buffer segments; and

determining, based on the at least one usage value for each logical data group of the plurality of logical data groups, a group delay threshold for each logical data group of the plurality of logical data groups, wherein determining the delay threshold for the at least one buffer segment includes selecting, based on the one logical data group corresponding to aggregate data elements in the at least one buffer segment, the delay threshold from a corresponding group delay threshold.

17. A system, comprising:

a data cache configured to aggregate a plurality of data requests in at least one buffer segment in the data cache;

a storage node configured to store aggregate data elements from the data cache in a persistent storage medium;

means for determining a percentage of writes to the persistent storage medium that meet a buffer threshold, wherein:

the buffer threshold is a predetermined amount of buffer space occupied by file data in the at least one buffer segment for a write operation to the persistent storage medium;

the at least one buffer segment has a serialized set of buffer positions;

the file data in the at least one buffer segment includes a plurality of file block sizes from the plurality of data requests; and

each file block of the file data in the at least one buffer segment is assigned to at least one buffer position using a caching algorithm;

means for determining, based on the percentage of writes, a delay threshold for the at least one buffer segment, wherein the delay threshold is a time value;

means for monitoring a commit time value for the at least one buffer segment after a first write request stores a first data element in the at least one buffer segment;

means for determining, based on the commit time value satisfying the delay threshold, a time-based data commit trigger for the at least one buffer segment, wherein the time-based data commit trigger is set independently of a current amount of buffer space occupied by aggregate data elements in the at least one buffer segment; and

means for moving, responsive to a first data commit trigger, aggregate data elements from the at least one buffer segment to the persistent storage medium.

18. The system of claim 17 , further comprising:

means for determining total writes to the persistent storage medium;

means for determining, among the total writes to the persistent storage medium, writes meeting the buffer threshold;

means for determining the percentage of writes is less than a usage target for the data cache, wherein the buffer threshold is based on a predetermined persistent data element size for the storage node; and

means for applying, responsive to determining the percentage of writes is less than the usage target, an adaptation algorithm to increase the time value for the delay threshold for the at least one buffer segment.

19. The system of claim 17 , wherein the means for determining the delay threshold is configured to:

determine a default delay threshold;

determine that the percentage of writes is less than a usage target; and

apply, responsive to determining that the percentage of writes is less than the usage target, an adaptation algorithm to the default delay threshold to increase the time value for the delay threshold.

20. The system of claim 19 , wherein the means for determining the delay threshold is further configured to:

iteratively apply, responsive to determining that the percentage of writes is less than the usage target, the adaptation algorithm to a prior delay threshold to determine the delay threshold.