Systems and methods for data migration in a clustered file system
Systems and methods for providing more efficient handling of I/O requests for clustered file system data subject to data migration. For instance, exemplary systems can more quickly determine if certain files on primary storage represent actual file data or stub data for recalling file data from secondary storage. Certain embodiments utilize a driver cache on each cluster node to maintain a record of recently accessed files that represent regular files (as opposed to stubs). A dual-locking process, using both strict locking and relaxed locking, maintains consistency between driver caches on different nodes and the data of the underlying clustered file system, while providing improved access to the data by the different nodes. Moreover, a signaling process can be used, such as with zero-length files, for alerting drivers on different nodes that data migration is to be performed and/or that the driver caches should be flushed.
1. A computer-implemented method of coordinating access to data in a data storage management system, the method comprising:
receiving a first request for a read-only lock on a signal file, wherein a locking status of the signal file indicates whether a migration operation is about to occur;
receiving a second request for the read-only lock on the signal file, wherein the first request and second request are not received contemporaneously;
in response to the first request for the read-only lock on the signal file being granted:
unlocking the signal file and re-requesting the read-only lock on the signal file after a predetermined period, and
operating in a first locking mode for requests to access a first file, the operating in the first locking mode comprising:
determining whether an associated driver cache includes information regarding the first file, and
in response to determining that the associated driver cache includes information regarding the first file, determining whether the first file is a stub file based on the associated driver cache; and
in response to the second request for the read-only lock on the signal file being denied:
clearing the associated driver cache storing information that indicates whether the first file stored on a primary storage device is a stub file, and
operating in a second locking mode for requests to access the first file, the operating in the second locking mode comprises refraining from using the associated driver cache to determine whether the first file is a stub file.
2. The method of claim 1 , wherein the first and second requests are requested by file system drivers of one or more computing devices.
3. The method of claim 2 , wherein the one or more computing devices is an independent node of a clustered file system.
4. The method of claim 1 , further comprising: in response to the second request for the read-only lock on the signal file being denied: releasing a second read-only lock on a token file.
5. The method of claim 4 , further comprising: performing the migration operation of the first file only after no second read-only locks remain on the token file.
6. The method of claim 4 , further comprising: obtaining with a migration service an exclusive lock on both the signal and token files prior to performing the migration operation.
7. The method of claim 1 , wherein the second locking mode is stricter than the first locking mode.
8. The method of claim 1 , further comprising: in response to a determining that the associated driver cache does not include information regarding the first file, analyzing a size or content of one or more inodes associated with the first file to determine whether the first file is a stub file.
9. The method of claim 1 , further comprising populating the associated driver cache with at least one or more inodes associated with the first file.
10. A data storage management system comprising:
one or more computing devices comprising one or more processors and computer memory, the one or more computing devices is configured to:
receive a first request for a read-only lock on a signal file, wherein a locking status of the signal file indicates whether a migration operation is about to occur;
in response to the first request for the read-only lock on the signal file being granted:
unlock the signal file and re-requesting the read-only lock on the signal file after a predetermined period, and
operate in a first locking mode for requests to access a first file, the operating in the first locking mode comprising:
determine whether an associated driver cache includes information regarding the first file, and
upon determining that the associated driver cache includes information regarding the first file, determine whether the first file is a stub file based on the associated driver cache;
receive a second request for the read-only lock on the signal file, wherein the first request and second request are not received contemporaneously; and
in response to the second request for the read-only lock on the signal file being denied:
clear the associated driver cache storing information that indicates whether the first file stored on a primary storage device is a stub file, and
operate in a second locking mode for requests to access the first file, comprising refraining from using the associated driver cache to determine whether the first file is a stub file.
11. The data storage management system in claim 10 , wherein the first and second requests are requested by file system drivers of the one or more computing devices.
12. The data storage management system in claim 11 , wherein the one or more computing devices is an independent node of a clustered file system.
13. The data storage management system in claim 10 , wherein the one or more computing devices is further configured to: release a second read-only lock on a token file in response to the second request for the read-only lock on the signal file being denied.
14. The data storage management system in claim 13 , wherein the one or more computing devices is further configured to: perform the migration operation of the first file only after no second read-only locks remain on the token file.
15. The data storage management system in claim 13 , wherein the one or more computing devices is further configured to: obtain, with a migration service, an exclusive lock on both the signal and token files prior to performing the migration operation.
16. The data storage management system in claim 10 , wherein the second locking mode is stricter than the first locking mode.
17. The data storage management system in claim 10 , wherein the one or more computing devices is further configured to: analyze a size of content of one or more inodes associated with the first file to determine whether the first file is a stub file in response to determining that the associated driver cache does not include information regarding the first file.
18. The data storage management system in claim 10 , wherein the one or more computing devices is further configured to: populate the associated driver cache with at least one or more inodes associated with the first file.