Systems and methods for managing delayed allocation on clustered file systems
The disclosed computer-implemented method for managing delayed allocation on clustered file systems may include (i) receiving, at a global lock manager that stores storage disk allocation information for a plurality of nodes in a clustered file system, a lock request from a node that requests a lock range on a storage disk to store data from a file, (ii) reserving, by the global lock manager, the lock range, (iii) receiving, at the global lock manager, from an additional node, an additional lock request for an additional lock range to store additional data from the file, and (iv) reserving, by the global lock manager, the additional lock range to be adjacent to the lock range on the storage disk based on the additional data on the additional node being from the same file as the data on the node. Various other methods, systems, and computer-readable media are also disclosed.
1. A computer-implemented method for managing delayed allocation on clustered file systems, at least a portion of the method being performed by a computing device comprising at least one processor, the method comprising:
receiving, at a global lock manager that stores, within a lock range data structure, storage disk allocation information for a plurality of nodes in a clustered file system, a lock request from a node that requests a lock range on a storage disk to store data from a file;
reserving, by the global lock manager, in the lock range data structure, the lock range on the storage disk for the data from the file in response to the lock request from the node;
receiving, at the global lock manager, from an additional node within the plurality of nodes in the clustered file system, an additional lock request for an additional lock range on the storage disk to store additional data from the file;
reserving, by the global lock manager, in the lock range data structure, the additional lock range to be adjacent to the lock range on the storage disk based on the additional data on the additional node being from the same file as the data on the node;
determining, at a quota management node, a delayed allocation quota for each individual node in the plurality of nodes, wherein the delayed allocation quota comprises a total amount of memory in a buffer available to be claimed by the individual node via lock ranges reserved by the individual node with the global lock manager;
determining, at the quota management node, that the node has met a delayed allocation quota for the node; and
disallowing, by the quota management node, the node from receiving additional lock ranges from the global lock manager until the node has written all of the data stored in the buffer for the node to the storage disk, wherein the data corresponds to the total amount of memory available in the buffer to be claimed by the node via the lock ranges.
2. The computer-implemented method of claim 1 , further comprising:
receiving, by the global lock manager, from the node, a query about the lock range for the data from the file; and
sending, by the global lock manager, to the node, information about the lock range.
3. The computer-implemented method of claim 2 , further comprising filling a section of the storage disk defined by the additional lock range with filler data in response to receiving the query about the lock range.
4. The computer-implemented method of claim 2 , further comprising sending, to the additional node, in response to receiving the query about the lock range, a message indicating that the additional node should write the additional data to a section of the storage disk defined by the additional lock range.
5. The computer-implemented method of claim 1 , further comprising:
receiving, at the node, a write request that comprises the data from the file;
storing, at the node, the data from the file in the buffer;
requesting, by the node, the lock range from the global lock manager;
determining, by the node, that the buffer needs to be cleared;
querying, by the node, the global lock manager about the lock range;
receiving, by the node, information about the lock range from the global lock manager; and
writing, by the node, the data from the file to a section of the storage disk defined by the lock range.
6. The computer-implemented method of claim 1 , further comprising:
storing, by the additional node, the additional data from the file in the buffer;
receiving, by the additional node, a message indicating that the additional node should write the additional data to a section of the storage disk defined by the additional lock range; and
writing, by the additional node, the additional data to the section of the storage disk defined by the additional lock range in response to the message despite having not determined, by the additional node, that the buffer needs to be cleared.
7. The computer-implemented method of claim 1 , further comprising:
determining, by the quota management node, that the node has written data to a previously requested lock range; and
allowing, by the quota management node, the node to receive the additional lock ranges.
8. The computer-implemented method of claim 1 , wherein the additional lock range is adjacent to the lock range on the storage disk when the additional lock range starts at a same memory address where the lock range ends.
9. A system for managing delayed allocation on clustered file systems, the system comprising:
a receiving module, stored in memory, that receives, at a global lock manager that stores, within a lock range data structure, storage disk allocation information for a plurality of nodes in a clustered file system, a lock request from a node that requests a lock range on a storage disk to store data from a file;
a reservation module, stored in memory, that reserves, by the global lock manager, in the lock range data structure, the lock range on the storage disk for the data from the file in response to the lock request from the node, wherein:
the receiving module receives, at the global lock manager, from an additional node within the plurality of nodes in the clustered file system, an additional lock request for an additional lock range on the storage disk to store additional data from the file; and
the reservation module reserves, by the global lock manager, in the lock range data structure, the additional lock range to be adjacent to the lock range on the storage disk based on the additional data on the additional node being from the same file as the data on the node; and
a quota management module, stored in memory, that:
determines, at a quota management node, a delayed allocation quota for each individual node in the plurality of nodes, wherein the delayed allocation quota comprises a total amount of memory available in a buffer to be claimed by the individual node via lock ranges reserved by the individual node with the global lock manager;
determines, at the quota management node, that the node has met a delayed allocation quota for the node; and
disallows, by the quota management node, the node from receiving additional lock ranges from the global lock manager until the node has written all of the data stored in the buffer for the node to the storage disk, wherein the data corresponds to the total amount of memory available in the buffer to be claimed by the node via the lock ranges; and
at least one physical hardware processor configured to execute the receiving module, the reservation module, and the quota management module.
10. The system of claim 9 , wherein:
the receiving module receives, by the global lock manager, from the node, a query about the lock range for the data from the file; and
the receiving module sends, by the global lock manager, to the node, information about the lock range.
11. The system of claim 10 , wherein the reservation module fills a section of the storage disk defined by the additional lock range with filler data in response to receiving the query about the lock range.
12. The system of claim 10 , wherein the receiving module sends, to the additional node, in response to receiving the query about the lock range, a message indicating that the additional node should write the additional data to a section of the storage disk defined by the additional lock range.
13. The system of claim 9 , further comprising:
a write receiving module, stored in memory, that receives, at the node, a write request that comprises the data from the file;
a buffer module, stored in memory, that stores, at the node, the data from the file in the buffer;
a lock range module, stored in memory, that requests, by the node, the lock range from the global lock manager; wherein:
the buffer module determines, by the node, that the buffer needs to be cleared;
the lock range module:
queries, by the node, the global lock manager about the lock range; and
receives, by the node, information about the lock range from the global lock manager; and
a writing module, stored in memory, that writes, by the node, the data from the file to a section of the storage disk defined by the lock range.
14. The system of claim 9 , further comprising:
a buffer module, stored in memory, that stores, by the additional node, the additional data from the file in the buffer;
a lock range module, stored in memory, that receives, by the additional node, a message indicating that the additional node should write the additional data to a section of the storage disk defined by the additional lock range; and
a writing module, stored in memory, that writes, by the additional node, the additional data to the section of the storage disk defined by the additional lock range in response to the message despite having not determined, by the additional node, that the buffer needs to be cleared.
15. The system of claim 9 , wherein the quota management module:
determines, by the quota management node, that the node has written data to a previously requested lock range; and
allows, by the quota management node, the node to receive the additional lock ranges.
16. The system of claim 9 , wherein the additional lock range is adjacent to the lock range on the storage disk when the additional lock range starts at a same memory address where the lock range ends.
17. A non-transitory computer-readable medium comprising one or more computer-readable instructions that, when executed by at least one processor of a computing device, cause the computing device to:
receive, at a global lock manager that stores, within a lock range data structure, storage disk allocation information for a plurality of nodes in a clustered file system, a lock request from a node that requests a lock range on a storage disk to store data from a file;
reserve, by the global lock manager, in the lock range data structure, the lock range on the storage disk for the data from the file in response to the lock request from the node;
receive, at the global lock manager, from an additional node within the plurality of nodes in the clustered file system, an additional lock request for an additional lock range on the storage disk to store additional data from the file;
reserve, by the global lock manager, in the lock range data structure, the additional lock range to be adjacent to the lock range on the storage disk based on the additional data on the additional node being from the same file as the data on the node;
determine, at a quota management node, a delayed allocation quota for each individual node in the plurality of nodes, wherein the delayed allocation quota comprises a total amount of memory in a buffer available to be claimed by the individual node via lock ranges reserved by the individual node with the global lock manager;
determine, at the quota management node, that the node has met a delayed allocation quota for the node; and
disallow, by the quota management node, the node from receiving additional lock ranges from the global lock manager until the node has written all of the data stored in the buffer for the node to the storage disk, wherein the data corresponds to the total amount of memory available in the buffer to be claimed by the node via the lock ranges.
18. The non-transitory computer-readable medium of claim 17 , wherein the one or more computer-readable instructions cause the computing device to:
receive, by the global lock manager, from the node, a query about the lock range for the data from the file; and
send, by the global lock manager, to the node, information about the lock range.
19. The non-transitory computer-readable medium of claim 18 , wherein the one or more computer-readable instructions cause the computing device to fill a section of the storage disk defined by the additional lock range with filler data in response to receiving the query about the lock range.
20. The non-transitory computer-readable medium of claim 18 , wherein the one or more computer-readable instructions cause the computing device to send, to the additional node, in response to receiving the query about the lock range, a message indicating that the additional node should write the additional data to a section of the storage disk defined by the additional lock range.