Enforcing consistency of file to storage node mapping
Systems and methods for enforcing consistency of file to storage node mappings in distributed file systems. An example method may include: receiving a request to perform a file operation with respect to a file associated with a volume of a distributed file system; identifying, among a plurality of storage nodes of the distributed file system, a storage node associated with the file; obtaining a lock of a layout of the volume; and responsive to determining, by a processing device, that a directory layout version number matches a volume layout version number, performing the requested file operation with respect to the file.
1. A method, comprising:
receiving a request to perform a file operation with respect to a file associated with a volume of a distributed file system;
identifying, among a plurality of storage nodes of the distributed file system, a storage node associated with the file;
identifying a directory layout version number reflecting a latest re-balancing operation with respect to the directory;
identifying a volume layout version number reflecting a latest modification of a layout of the volume; and
obtaining a lock of a layout of the volume;
responsive to determining, by a processing device, that the directory layout version number matches the volume layout version number, performing the requested file operation with respect to the file.
2. The method of claim 1 , further comprising:
releasing the lock of the volume.
3. The method of claim 1 , wherein the directory layout version number is specified by the request to perform the file operation.
4. The method of claim 1 , wherein identifying the storage node associated with the file comprises:
determining a value of a pre-defined hash function of a name of the file; and
identifying, among a plurality of storage nodes of the distributed file system, a storage node associated with a hash range to which the value belongs.
5. The method of claim 4 , wherein identifying the storage node associated with a hash range to which the value belongs is performed in view of a directory-specific hash to storage node mapping.
6. The method of claim 1 , further comprising:
associating a range of hash values with each storage node of a cluster of storage nodes.
7. The method of claim 1 , wherein obtaining the lock of the layout of the volume is performed with respect to a storage node associated with a hash range to which a value of a pre-defined hash function of a name of the file belongs.
8. A system, comprising:
a memory; and
a processing device, coupled to the memory, to:
receive a request to perform a file operation with respect to a file associated with a volume of a distributed file system;
identify, among a plurality of storage nodes of the distributed file system, a storage node associated with the file;
obtain a lock of a layout of the volume; and
responsive to determining that a directory layout version number matches a volume layout version number, perform the requested file operation with respect to the file.
9. The system of claim 8 , wherein the processing device is further to:
release the lock of the volume.
10. The system of claim 8 , wherein the directory layout version number is specified by the request to perform the file operation.
11. The system of claim 8 , wherein to identify the storage node associated with the file, the processing device is further to:
determine a value of a pre-defined hash function of a name of the file; and
identify, among a plurality of storage nodes of the distributed file system, a storage node associated with a hash range to which the value belongs.
12. The system of claim 8 , wherein the processing device is to obtain the lock of the layout of the volume with respect to a storage node associated with a hash range to which a value of a pre-defined hash function of a name of the file belongs.
13. A non-transitory computer-readable storage medium comprising executable instructions that, when executed by a processing device, cause the processing device to:
receive a request to perform a file operation with respect to a file associated with a volume of a distributed file system;
determine a value of a pre-defined hash function of a name of the file; and
identify, among a plurality of storage nodes of the distributed file system, a storage node associated with a hash range to which the value belongs;
obtain a lock of a layout of the volume; and
responsive to determining, by the processing device, that a directory layout version number matches a volume layout version number, perform the requested file operation with respect to the file.
14. The non-transitory computer-readable storage medium of claim 13 , further comprising executable instructions that cause the processing device to:
release the lock of the volume.
15. The non-transitory computer-readable storage medium of claim 13 , wherein the directory layout version number is specified by the request to perform the file operation.
16. The non-transitory computer-readable storage medium of claim 13 , wherein obtaining the lock of the layout of the volume is performed with respect to a storage node associated with a hash range to which a value of a pre-defined hash function of a name of the file belongs.