Compounding file system metadata operations via buffering

1. A method of processing file system operations, comprising:

receiving, by one or more processors associated with a node, an indication to send locally-stored file system operations to a remote file system metadata server, wherein the node is connected to the remote file system metadata server via one or more networks, and the indication corresponds to an indication of a lease break with respect to at least part of a lease associated with the node;

determining, by one or more processors associated with the node, a linear chain of dependent file system operations buffered in a write buffer at the node based at least in part on a dependency graph or other data structure reflecting dependencies between the locally-stored file system operations, wherein:

a kernel module of an operating system associated with a node intercepts file system operations affecting data subject to the lease;

the file system operations that are intercepted are buffered in the write buffer; and

determining the linear chain of dependent file system operations buffered in the write buffer comprises: determining a sequence of operations that has at most a single dependency on a next prior operation in the corresponding linear chain;

combining, by one or more processors associated with the node, at least a subset of operations comprising the linear chain into a compound file system operation, wherein the at least the subset of operations that are combined into the compound file system comprise a plurality of operations buffered in the write buffer at the node with respect to at least the part of the lease corresponding to the lease break, wherein buffering the file system operations in the write buffer includes reducing resource consumption and communication latency by compounding the combined operations into a single data packet; and

sending, by one or more processors associated with the node, the compound file system operation to the remote file system metadata server.

2. The method of claim 1 , wherein each operation in the linear chain has inbound dependencies from at most one other operation in the chain and outbound dependencies on at most one other operation in the chain.

3. The method of claim 1 , wherein the linear chain comprises a linear subgraph of said dependency graph or other data structure.

4. The method of claim 1 , wherein the locally-stored file system operations comprise file system operations affecting file system metadata.

5. The method of claim 1 , wherein the locally-stored file system operations comprise file system operations that have been buffered locally to permit asynchronous transmission to the remote file system metadata server.

6. The method of claim 5 , wherein the lease comprises a metadata write lease, and the locally-stored file system operations are associated with a portion of file system metadata with respect to which the metadata write lease is held.

7. The method of claim 6 , wherein the indication to send the locally-stored file system operations to the remote file system metadata server comprises a lease break communication with respect to the metadata write lease.

8. The method of claim 1 , wherein sending the compound file system operation to the remote file system metadata server comprises including the compound file system operation in a single package and sending the single package to the file system metadata server.

9. A system, comprising:

a communication interface; and

one or more processors coupled to the communication interface and configured to:

receive an indication to send locally-stored file system operations to a remote file system metadata server, wherein the system is connected to the remote file system metadata server via one or more networks, and the indication corresponds to an indication of a lease break with respect to at least part of a lease associated with the node;

determine a linear chain of dependent file system operations buffered in a write buffer at the system based at least in part on a dependency graph or other data structure reflecting dependencies between the locally-stored file system operations, wherein:

a kernel module of an operating system associated with a node intercepts file system operations affecting data subject to the lease;

the file system operations that are intercepted are buffered in the write buffer; and

to determine the linear chain of dependent file system operations buffered in the write buffer comprises: determining a sequence of operations that has at most a single dependency on a next prior operation in the corresponding linear chain;

combine at least a subset of operations comprising the linear chain into a compound file system operation, wherein the at least the subset of operations that are combined into the compound file system comprise a plurality of operations buffered in the write buffer at the node with respect to at least the part of the lease corresponding to the lease break, wherein buffering the file system operations in the write buffer includes reducing resource consumption and communication latency by compounding the combined operations into a single data packet; and

send the compound file system operation to the remote file system metadata server via the communication interface.

10. The system of claim 9 , further comprising a memory configured to store the dependency graph.

11. The system of claim 9 , wherein each operation in the linear chain has inbound dependencies from at most one other operation in the chain and outbound dependencies on at most one other operation in the chain.

12. The system of claim 9 , wherein the linear chain comprises a linear subgraph of said dependency graph or other data structure.

13. The system of claim 9 , wherein the locally-stored file system operations comprise file system operations that have been buffered locally to permit asynchronous transmission to the remote file system metadata server.

14. The system of claim 13 , wherein the lease comprises a metadata write lease, and the locally-stored file system operations are associated with a portion of file system metadata with respect to which the metadata write lease is held.

15. The system of claim 14 , wherein the indication to send the locally-stored file system operations to the remote file system metadata server comprises a lease break communication with respect to the metadata write lease.

16. The system of claim 9 , wherein sending the compound file system operation to the remote file system metadata server comprises including the compound file system operation in a single package and sending the single package to the file system metadata server.

17. A computer program product to process file system operations, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:

receiving, by one or more processors associated with a node, an indication to send locally-stored file system operations to a remote file system metadata server, and the indication corresponds to an indication of a lease break with respect to at least part of a lease associated with the node;

determining, by one or more processors associated with the node, a linear chain of dependent file system operations buffered in a write buffer at the node based at least in part on a dependency graph or other data structure reflecting dependencies between the locally-stored file system operations, wherein:

a kernel module of an operating system associated with a node intercepts file system operations affecting data subject to the lease;

the file system operations that are intercepted are buffered in the write buffer; and

determining the linear chain of dependent file system operations buffered in the write buffer comprises: determining a sequence of operations that has at most a single dependency on a next prior operation in the corresponding linear chain;

combining, by one or more processors associated with the node, at least a subset of operations comprising the linear chain into a compound file system operation, wherein the at least the subset of operations that are combined into the compound file system comprise a plurality of operations buffered in the write buffer at the node with respect to at least the part of the lease corresponding to the lease break, wherein buffering the file system operations in the write buffer includes reducing resource consumption and communication latency by compounding the combined operations into a single data packet; and

sending, by one or more processors associated with the node, the compound file system operation to the remote file system metadata server.

18. The computer program product of claim 17 , wherein the locally-stored file system operations comprise file system operations that have been buffered locally to permit asynchronous transmission to the remote file system metadata server.

19. The computer program product of claim 18 , wherein the lease comprises a metadata write lease, and the locally-stored file system operations are associated with a portion of file system metadata with respect to which the metadata write lease is held.

20. The computer program product of claim 19 , wherein the indication to send the locally-stored file system operations to the remote file system metadata server comprises a lease break communication with respect to the metadata write lease.

21. The method of claim 1 , further comprising:

determining, by one or more processors associated with the node, the plurality of operations buffered at the node with respect to at least the part of the lease corresponding to the lease break, wherein the plurality of operations are determined based on the indication of the lease break.

22. The method of claim 1 , wherein the plurality of operations correspond to a plurality of operations buffered at the node that affect information stored at the server which is subject to the lease.

23. The method of claim 1 , wherein the plurality of operations correspond to a plurality of operations buffered at the node that affect information at the server which is subject to the lease break.

24. The method of claim 1 , wherein none of the operations in the sequence of operations has more than one inbound dependency.

25. The method of claim 7 , wherein the lease break communication is received from the remote file system metadata server.

26. The method of claim 1 , wherein the at least the subset of operations comprising the linear chain into the compound file system operation are combined in a manner that the compound file system operation is communicated to the remote file system metadata server in a single communication.

27. The method of claim 1 , wherein the determining a linear chain of dependent file system operations buffered in a write buffer at the node based at least in part on a dependency graph or other data structure reflecting dependencies between the locally-stored file system operations comprises:

determining the dependency graph or other data structure reflecting dependencies between the locally-stored file system operations;

determining, based on the dependencies between the locally-stored file system operations using the dependency graph or other data structure, the sequence of operations in the dependency graph or other data structure such that the sequence of operations have at most a single dependency on a next prior operation in the sequence of operations; and

determining a linear chain of operations to be combined into the compound file system operation based on a determination of the sequence of operations in the dependency graph or other data structure having no more than a single dependency on the next prior operation in the sequence of operations.