Concurrent access and transactions in a distributed file system
Embodiments described herein provide techniques for maintaining consistency in a distributed system (e.g., a distributed secondary storage system). According to one embodiment of the present disclosure, a first set of file system objects included in performing the requested file system operation is identified in response to a request to perform a file system operation. An update intent corresponding to the requested file system operation is inserted into an inode associated with each identified file system object. Each file system object corresponding to the inode is modified as specified by the update intent in that inode. After modifying the file system object corresponding to the inode, the update intent is removed from that inode.
1. A method, comprising:
obtaining locks for a plurality of file system objects associated with a file system transaction, wherein the plurality of file system objects correspond to inodes;
storing a corresponding update intent in each of the file system objects associated with the file system transaction, wherein the corresponding update intent is stored in the inodes corresponding to the file system objects associated with the file system transaction, wherein the corresponding update intent indicates an order in which a plurality of file system objects associated with the file system transaction are to be modified; and
modifying the plurality of file system objects associated with the file system transactions based on the order;
removing the corresponding update intent from a file system object of the plurality of file system objects after the file system object is modified; and
releasing the locks for the plurality of file system objects associated with the file system transaction.
2. The method of claim 1 , further comprising generating the corresponding update intent for each file system object associated with the file system transaction.
3. The method of claim 1 , wherein the corresponding update intent indicates a file system operation to be performed for the file system transaction.
4. The method of claim 1 , further comprising receiving at a node of a distributed file system, a request to perform a file system operation.
5. The method of claim 4 , wherein obtaining locks for the plurality of file system objects associated with the file system transaction includes identifying the plurality of file system objects associated with the file system operation.
6. The method of claim 5 , wherein obtaining locks for the plurality of file system objects associated with the file system transaction includes requesting by the node from a distributed ticket service corresponding tickets for the plurality of identified file system objects.
7. The method of claim 6 , wherein the distributed ticket service determines whether the requested tickets are currently being held by one or more other nodes of the distributed file system.
8. The method of claim 7 , wherein in the event the requested tickets are currently being held by the one or more other nodes of the distributed file system, the requesting node waits until the requested tickets are no longer held by the one or more other nodes to receive the requested tickets.
9. The method of claim 7 , wherein obtaining locks for the plurality of file system objects associated with the file system transaction includes receiving the requested tickets from the distributed ticket service.
10. The method of claim 9 , wherein obtaining locks for the plurality of file system objects associated with the file system transaction includes requesting a process-wide lock for each of the file system objects.
11. The method of claim 10 , further comprising performing the file system operation.
12. A computer program product, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
obtaining locks for a plurality of file system objects associated with a file system transaction, wherein the plurality of file system objects correspond to inodes;
storing a corresponding update intent in each of the file system objects associated with the file system transaction, wherein the corresponding update intent is stored in the inodes corresponding to the file system objects associated with the file system transaction, wherein the corresponding update intent indicates an order in which a plurality of file system objects associated with the file system transaction are to be modified; and
modifying the plurality of file system objects associated with the file system transactions based on the order;
removing the corresponding update intent from a file system object of the plurality of file system objects after the file system object is modified; and
releasing the locks for the plurality of file system objects associated with the file system transaction.
13. The computer program product of claim 12 , further comprising instructions for generating the corresponding update intent for each of the file system objects associated with the file system transaction.
14. The computer program product of claim 12 , wherein the corresponding update intent indicates a file system operation to be performed for the file system transaction.
15. The computer program product of claim 12 , wherein obtaining locks for the plurality of file system objects associated with the file system transaction includes:
receiving at a node of a distributed file system, a request to perform a file system operation; and
identifying a plurality of file system objects associated with the file system operation.
16. The computer program product of claim 15 , wherein obtaining locks for the plurality of file system objects associated with the file system transaction includes:
requesting by the node from a distributed ticket service corresponding tickets for the plurality of identified file system objects; and
receiving the requested tickets from the distributed ticket service.
17. The computer program product of claim 16 , wherein obtaining locks for the plurality of file system objects associated with the file system transaction includes:
requesting a process-wide lock for each of the file system objects; and
performing the file system operation.
18. A system comprising:
a processor; and
a memory coupled with the processor, wherein the memory is configured to provide the processor with instructions which when executed cause the processor to:
obtain locks for a plurality of file system objects associated with a file system transaction, wherein the plurality of file system objects correspond to inodes;
store a corresponding update intent in each of the file system objects associated with the file system transaction, wherein the corresponding update intent is stored in the inodes corresponding to the file system objects associated with the file system transaction, wherein the corresponding update intent indicates an order in which a plurality of file system objects associated with the file system transaction are to be modified;
modify the plurality of file system objects associated with the file system transactions based on the order;
remove the corresponding update intent from a file system object of the plurality of file system objects after the file system object is modified; and
release the locks for the plurality of file system objects associated with the file system transaction.