Enforcing consistency of file to storage node mapping
Systems and methods for enforcing consistency of file to storage node mappings in distributed file systems. An example method may include: receiving a request to perform a file operation with respect to a file associated with a directory of a distributed file system; identifying a directory layout version number associated with the directory; identifying a volume layout version number associated with a volume in which the directory resides; and responsive to determining that the directory layout version number matches the volume layout version number, performing the requested file operation with respect to the file.
1. A method, comprising:
receiving, by a processing device, a request to perform a file operation with respect to a file associated with a directory of a distributed file system;
identifying a directory layout version number assigned to the directory;
identifying a volume layout version number assigned to a volume in which the directory resides; and
responsive to determining that the directory layout version number matches the volume layout version number, performing the requested file operation with respect to the file.
2. The method of claim 1 , wherein the directory layout version number is specified by the request to perform the file operation.
3. The method of claim 1 , wherein performing the requested file operation further comprises:
obtaining a lock of a layout of the volume.
4. The method of claim 3 , further comprising:
releasing the lock of the layout of the volume responsive to completing the requested file operation.
5. The method of claim 1 , wherein performing the requested file operation further comprises:
identifying a storage node storing contents of the file.
6. The method of claim 1 , wherein the volume layout version number is stored on a storage node storing contents of the file.
7. The method of claim 1 , wherein performing the requested file operation further comprises:
determining a value of a hash function of a name of the file; and
identifying, among a plurality of storage nodes of the distributed file system, a storage node associated with a hash range to which the value belongs.
8. The method of claim 7 , further comprising:
associating a range of hash values with each storage node of the distributed file system;
migrating, in view of the associating operation, a file from a first storage node of the distributed file system to a second storage node of the distributed file system; and
incrementing the directory layout version number.
9. The method of claim 1 , further comprising:
adding a storage node to the distributed file system; and
associating a range of hash values with each storage node of the distributed file system;
migrating, in view of the associating operation, a file from a first storage node of the distributed file system to a second storage node of the distributed file system; and
incrementing the volume layout version number.
10. The method of claim 1 , further comprising:
responsive to determining that the directory layout version number does not match the volume layout version number, rejecting the requested file operation.
11. The method of claim 1 , wherein the directory spans over two or more storage nodes of the distributed file system.
12. A system, comprising:
a memory; and
a processing device, coupled to the memory, wherein the processing device is configured to:
receive a request to perform a file operation with respect to a file associated with a directory of a distributed file system, wherein the request identifies a directory layout version number assigned to the directory;
identify a volume layout version number assigned to a volume in which the directory resides; and
responsive to determining that the directory layout version number matches the volume layout version number, performing the requested file operation with respect to the file.
13. The system of claim 12 , wherein performing the requested file operation further comprises:
obtaining a lock of a layout of the volume.
14. The system of claim 13 , wherein the processing device is further to:
release the lock of the layout of the volume responsive to completing the requested file operation.
15. The system of claim 12 , wherein the volume layout version number is stored on a storage node storing contents of the file.
16. The system of claim 12 , wherein performing the requested file operation further comprises:
determining a value of a hash function of a name of the file; and
identifying, among a plurality of storage nodes of the distributed file system, a storage node associated with a hash range to which the value belongs.
17. A non-transitory computer-readable storage medium comprising executable instructions that, when executed by a processing device, cause the processing device to:
receive a request to perform a file operation with respect to a file associated with a directory of a distributed file system, wherein the request identifies a directory layout version number assigned to the directory;
identify a volume layout version number assigned to a volume in which the directory resides; and
responsive to determining that the directory layout version number matches the volume layout version number, performing the requested file operation with respect to the file.
18. The non-transitory computer-readable storage medium of claim 17 , wherein performing the requested file operation further comprises:
obtaining a lock of a layout of the volume.
19. The non-transitory computer-readable storage medium of claim 17 , wherein the volume layout version number is stored on a storage node storing contents of the file.
20. The non-transitory computer-readable storage medium of claim 17 , wherein performing the requested file operation further comprises:
determining a value of a hash function of a name of the file; and
identifying, among a plurality of storage nodes of the distributed file system, a storage node associated with a hash range to which the value belongs.