Distributed management of file modification-time field
A method for data storage includes assigning multiple different portions of a given object, which is stored in non-volatile storage, for access by multiple respective software modules running on one or more processors. Storage operations are performed by the software modules in the portions of the given object. Respective local values, which are indicative of most recent times the storage operations were performed in the given object by the respective software modules, are updated by the software modules. A global value is updated for the given object, by consolidating updates of the local values performed by the software modules.
1. A computer-implemented method executing on data processing hardware that causes the data processing hardware to perform operations comprising:
obtaining a directory modification time value associated with a directory comprising a plurality of files, the directory modification time value representing a most recent file modification time value of the plurality of files;
transmitting one or more files of the plurality of files to each of a plurality of servers, each of the plurality of servers maintaining a file modification time value for each of the transmitted one or more files of the plurality of files;
in response to one of the servers of the plurality of servers modifying a respective file of the transmitted one or more files of the plurality of files, updating the directory modification time value associated with the directory and storing a portion of the modified respective file with the updated directory modification time value;
receiving, from an application, a request for the respective file of the transmitted one or more files of the plurality of files; and
based on the request, transmitting, to the application, the updated directory modification time value and the portion of the modified respective file.
2. The method of claim 1 , wherein each of the plurality of servers executes a Network File System (NFS).
3. The method of claim 1 , wherein each of the plurality of servers maintains a local modification time value for each of the transmitted one or more plurality of files.
4. The method of claim 1 , wherein the operations further comprise, upon recovery from a failure:
determining a count of currently-open write operations to the plurality of files; and
updating the directory modification time value associated with the directory based on the count of currently-open write operations.
5. The method of claim 1 , wherein updating the directory modification time value associated with the directory comprises:
receiving, from the one of the servers of the plurality of servers, a request to update the directory modification time value, the request to update the directory modification time value comprising the file modification time value corresponding to the modified file;
determining that the corresponding file modification time value is greater than the directory modification time value; and
based on determining that the corresponding file modification time value is greater than the directory modification time value, replacing the directory modification time value with the corresponding file modification time value.
6. The method of claim 1 , wherein updating the directory modification time value associated with the directory comprises:
receiving, from the one of the servers of the plurality of servers, a request to update the directory modification time value, the request to update the directory modification time value comprising the file modification time value corresponding to the modified file;
determining that the corresponding file modification time value is less than or equal to the directory modification time value; and
based on determining that the corresponding file modification time value is less than or equal to the directory modification time value, incrementing the directory modification time value by a predefined increment.
7. The method of claim 6 , wherein the predefined increment comprises a finest-resolution increment in a format of the directory modification time value.
8. The method of claim 6 , wherein the operations further comprise, upon recovery from a failure, updating the directory modification time value based on the predefined increment.
9. The method of claim 1 , wherein the operations further comprise executing a distributed file system.
10. The method of claim 1 , wherein each of the plurality of servers execute a Server Message Block (SMB) protocol.
11. A system comprising:
data processing hardware; and
memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to perform operations comprising:
obtaining a directory modification time value associated with a directory comprising a plurality of files, the directory modification time value representing a most recent file modification time value of the plurality of files;
transmitting one or more files of the plurality of files to each of a plurality of servers, each of the plurality of servers maintaining a file modification time value for each of the transmitted one or more files of the plurality of files;
in response to one of the servers of the plurality of servers modifying a respective file of the transmitted one or more files of the plurality of files, updating the directory modification time value associated with the directory and storing a portion of the modified respective file with the updated directory modification time value;
receiving, from an application, a request for the respective file of the transmitted one or more files of the plurality of files; and
based on the request, transmitting, to the application, the updated directory modification time value and the portion of the modified respective file.
12. The system of claim 11 , wherein each of the plurality of servers executes a Network File System (NFS).
13. The system of claim 11 , wherein each of the plurality of servers maintains a local modification time value for each of the transmitted one or more plurality of files.
14. The system of claim 11 , wherein the operations further comprise, upon recovery from a failure:
determining a count of currently-open write operations to the plurality of files; and
updating the directory modification time value associated with the directory based on the count of currently-open write operations.
15. The system of claim 11 , wherein updating the directory modification time value associated with the directory comprises:
receiving, from the one of the servers of the plurality of servers, a request to update the directory modification time value, the request to update the directory modification time value comprising the file modification time value corresponding to the modified file;
determining that the corresponding file modification time value is greater than the directory modification time value; and
based on determining that the corresponding file modification time value is greater than the directory modification time value, replacing the directory modification time value with the corresponding file modification time value.
16. The system of claim 11 , wherein updating the directory modification time value associated with the directory comprises:
receiving, from the one of the servers of the plurality of servers, a request to update the directory modification time value, the request to update the directory modification time value comprising the file modification time value corresponding to the modified file;
determining that the corresponding file modification time value is less than or equal to the directory modification time value; and
based on determining that the corresponding file modification time value is less than or equal to the directory modification time value, incrementing the directory modification time value by a predefined increment.
17. The system of claim 16 , wherein the predefined increment comprises a finest-resolution increment in a format of the directory modification time value.
18. The system of claim 16 , wherein the operations further comprise, upon recovery from a failure, updating the directory modification time value based on the predefined increment.
19. The system according to claim 11 , wherein the operations further comprise executing a distributed file system.
20. The system according to claim 11 , wherein each of the plurality of servers execute a Server Message Block (SMB) protocol.