Concurrent access and transactions in a distributed file system
According to one embodiment of the present disclosure, a first set of file system objects included in performing a requested file system operation is identified in response to a request to perform a file system operation. An update intent corresponding to the requested file system operation is inserted into a data structure associated with each identified file system object. Each file system object corresponding to the corresponding data structure is modified as specified by the update intent in that data structure. After modifying the file system object corresponding to the corresponding data structure, the update intent is removed from that data structure.
1. A computer implemented method, comprising:
in response to a request to perform a file system operation, identifying a set of file system objects to be accessed in performing the file system operation, wherein each file system object of the set of file system objects is associated with a corresponding file system data structure in a set of file system data structures, wherein identifying the set of file system objects to be accessed in performing the file system operation includes determining whether any file system data structure included in the set of file system data structures includes a previous update intent associated with a previous file system operation;
for the requested file system operation, storing a corresponding update intent in each file system data structure in the set of file system data structures, wherein the corresponding update intent specifies an order list of the file system data structures in the set of file system data structures;
performing a part of the file system operation, wherein the part of the file system operation is associated with the file system data structure corresponding to a selected position in the order list; and
in response to determining that the part of the file system operation has been successfully completed for the file system data structure in the selected position in the order list, removing the corresponding update intent from the file system data structure corresponding to the selected position in the order list.
2. The computer implemented method of claim 1 , wherein the set of file system objects includes a directory.
3. The computer implemented method of claim 1 , wherein the set of file system objects includes a file.
4. The computer implemented method of claim 1 , wherein the set of file system data structures includes inodes.
5. The computer implemented method of claim 1 , further comprising in response to a determination that the set of file system data structures includes the previous update intent associated with the previous file system operation, determining whether to complete or abandon an incomplete file system operation.
6. The computer implemented method of claim 5 , wherein the determination of whether to complete or abandon the incomplete file system operation is based at least in part on which file system data structure of a second set of file system data structures identified by the previous update intent includes the previous update intent associated with the previous file system operation.
7. The computer implemented method of claim 5 , wherein in response to a determination to abandon the incomplete file system operation, removing the previous update intent associated with the previous file system operation from a second set of file system data structures associated with the previous update intent.
8. The computer implemented method of claim 5 , wherein in response to a determination to complete the incomplete file system operation, completing each part of the previous file system operation corresponding to each file system data structure of a second set of file system data structures that includes the previous update intent.
9. The computer implemented method of claim 8 , further comprising removing the previous update intent from each file system data structure of the second set of file system data structures after completion of a corresponding part of the previous file system operation.
10. The computer implemented method of claim 1 , wherein performing the part of the file system operation includes modifying the order list.
11. The computer implemented method of claim 1 , wherein prior to storing the corresponding update intent in each file system data structure in the set of file system data structures, obtaining a node-specific lock and a process-specific lock on each file system data structure in the set of file system data structures to be modified.
12. The computer implemented method of claim 1 , wherein the method is performed using a distributed file system.
13. The computer implemented method of claim 12 , wherein the distributed file system is exposed to a client as an NFS (Network File System), SMB (Server Message Block), or CIFS (Common Internet File System) mount point, and the requested file system operation is an NFS, SMB, or CIFS operation sent from the client to a node of the distributed file system.
14. A system, comprising:
a processor configured to:
in response to a request to perform a file system operation, identify a set of file system objects to be accessed in performing the file system operation, wherein each file system object of the set of file system objects is associated with a corresponding file system data structure in a set of file system data structures, wherein the processor is configured to identify the set of file system objects to be accessed in performing the file system operation including by being configured to determine whether any file system data structure included in the set of file system data structures includes a previous update intent associated with a previous file system operation;
for the requested file system operation, store a corresponding update intent in each file system data structure in the set of file system data structures, wherein the corresponding update intent specifies an order list of the file system data structures in the set of file system data structures;
perform a part of the file system operation, wherein the part of the file system operation is associated with the file system data structure corresponding to a selected position in the order list; and
in response to determining that the part of the file system operation has been successfully completed for the file system data structure in the selected position in the order list, remove the corresponding update intent from the file system data structure corresponding to the selected position in the order list; and
a memory coupled to the processor and configured to provide the processor with instructions.
15. The system of claim 14 , wherein the processor is further configured to: in response to a determination that the set of file system data structures includes the previous update intent associated with the previous file system operation, determine whether to complete or abandon an incomplete file system operation.
16. The system of claim 15 , wherein the determination of whether to complete or abandon the incomplete file system operation is based at least part on which file system data structure of a second set of file system data structures includes the previous update intent associated with the previous file system operation.
17. The system of claim 14 , wherein the system is included in a distributed file system.
18. A computer program product, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
in response to a request to perform a file system operation, identifying a set of file system objects to be accessed in performing the file system operation, wherein each file system object of the set of file system objects is associated with a corresponding file system data structure in a set of file system data structures, wherein identifying the set of file system objects to be accessed in performing the file system operation includes determining whether any file system data structure included in the set of file system data structures includes a previous update intent associated with a previous file system operation;
for the requested file system operation, storing a corresponding update intent in each file system data structure in the set of file system data structures, wherein the corresponding update intent specifies an order list of the file system data structures in the set of file system data structures;
performing a part of the file system operation, wherein the part of the file system operation is associated with the file system data structure corresponding to a selected position in the order list; and
in response to determining that the part of the file system operation has been successfully completed for the file system data structure in the selected position in the order list, removing the corresponding update intent from the file system data structure corresponding to the selected position in the order list.