Remote file virtualization in a switched file system
View Patent ↗A plurality of network file manager switches interoperate to provide remote file virtualization. Copies of file data and/or metadata are maintained at a central site and at one or more remote sites. The network file manager switch at the remote site may satisfy certain client requests locally without having to contact the network file manager switch at the central site. A global namespace is maintained and is communicated to all network file manager switches.
1. A central network file manager apparatus comprising:
one or more processors; and
memory coupled to the one or more processors and configured to store an authoritative copy of one or more files selected from at least one of data files or metadata files and wherein the one or more processors are configured to execute program instructions stored in the memory comprising:
communicating mirror copies of each authoritative copy of the one or more files to each of a plurality of remote network file managers;
obtaining updated file data from one of the plurality of remote network file managers when one of the mirror copies is updated by a client at the one remote network file manager;
updating the authoritative copy corresponding to the updated mirror copy with the updated file data;
notifying each of the other remote network file managers that the respective mirror copy stored thereon, and corresponding to the updated mirror copy, is no longer valid; and
asynchronously communicating a mirror copy of the updated authoritative copy to the other remote network file managers using a lazy mirroring technique based on available network bandwidth.
2. An apparatus according to claim 1 , wherein the central network file manager and the plurality of remote network file managers maintain a common global namespace.
3. An apparatus according to claim 1 , wherein the communicating mirror copies of each authoritative copy program instructions further comprise asynchronously pushing each copy to each remote network file manager using a lazy mirroring technique based on available network bandwidth.
4. An apparatus according to claim 3 , wherein the one or more processors are further configured to execute programmed instructions stored in the memory further comprising:
after pushing each mirror copy to each remote network file manager, verifying that the data has not changed since being pushed; and
notifying each remote network file manager that the data is valid.
5. An apparatus according to claim 1 , wherein the communicating mirror copies of each authoritative copy program instructions further comprise asynchronous responding, using a lazy mirroring technique based on available network bandwidth, to a pull request for a mirror copy from one or more of the remote network file managers.
6. An apparatus according to claim 5 , wherein the one or more processors are further configured to execute programmed instructions stored in the memory further comprising after communicating one or more mirror copies to one or more remote network file managers in response to a pull request, receiving a request, from each of the one or more remote network file managers, to verify that the data is still valid.
7. An apparatus according to claim 1 , wherein the one or more processors are further configured to execute programmed instructions stored in the memory further comprising:
maintaining statistics regarding access patterns by remote clients; and
asynchronously communicating a mirror copy of the updated authoritative copy to the other remote network file managers based on the statistics.
8. An apparatus according to claim 1 , wherein the authoritative copies are mirrored in at least one of a breadth-first fashion or a depth-first fashion.
9. An apparatus according to claim 1 , wherein the communicating mirror copies of each authoritative copy program instructions further comprise asynchronously communicating each copy to each remote network file manager based on at least one of a first access to the authoritative copy by one or more of the remote network file managers, pre-fetching based on one or more authoritative copy access characteristic, and pre-loading of the authoritative copies prior to instantiation of one or more remote network file managers in the network at a remote site.
10. An apparatus according to claim 1 wherein the one or more processors are further configured to execute programmed instructions stored in the memory further comprising communicating with a wide area network (WAN) optimization apparatus that operates as a client to file server nodes, wherein the apparatus implements server message block (SMB) signing on communications with the file server nodes including SMB signing on messages used to pre-fetch data from the file server nodes.
11. A method for managing files, comprising:
storing at a central network file manager, an authoritative copy of one or more files selected from at least one of data files or metadata files;
communicating with the central network file manager mirror copies of each authoritative copy to each of a plurality of remote network file managers;
obtaining at the central network file manager, updated file data from one of the plurality of remote network file managers when one of the mirror copies is updated by a client at the one remote network file manager;
updating at the central network file manager the authoritative copy corresponding to the updated mirror copy with the updated file data;
notifying with the central network file manager each of the other remote network file managers that the respective mirror copy stored thereon, and corresponding to the updated mirror copy, is no longer valid; and
asynchronously communicating with the central network file manager a mirror copy of the updated authoritative copy to the other remote network file managers using a lazy mirroring technique based on available network bandwidth.
12. A method according to claim 11 , wherein the central network file manager and the plurality of remote network file managers maintain a common global namespace.
13. A method according to claim 11 , wherein the communicating mirror copies of each authoritative copy further comprises asynchronously pushing each copy to each remote network file manager using a lazy mirroring technique based on available network bandwidth.
14. A method according to claim 13 , further comprising:
after pushing each mirror copy to each remote network file manager, verifying that the data has not changed since being pushed; and
notifying each remote network file manager that the data is valid.
15. A method according to claim 11 , wherein the communicating mirror copies of each authoritative copy further comprises asynchronous responding, using a lazy mirroring technique based on available network bandwidth, to a pull request for a mirror copy from one or more of the remote network file managers.
16. A method according to claim 15 further comprising after communicating one or more mirror copies to one or more remote network file managers in response to a pull request, receiving a request, from each of the one or more remote network file managers, to verify that the data is still valid.
17. A method according to claim 11 , further comprising:
maintaining statistics with the central file manager apparatus regarding access patterns by remote clients; and
asynchronously communicating with the central file manager apparatus a mirror copy of the updated authoritative copy to the other remote network file managers based on the statistics.
18. A method according to claim 11 , wherein the authoritative copies are mirrored in at least one of a breadth-first fashion or a depth-first fashion.
19. A method according to claim 11 , wherein the communicating mirror copies of each authoritative copy further comprises asynchronously communicating each copy to each remote network file manager based on at least one of a first access to the authoritative copy by one or more of the remote network file managers, pre-fetching based on one or more authoritative copy access characteristic, and pre-loading of the authoritative copies prior to instantiation of one or more remote network file managers in the network at a remote site.
20. A method according to claim 11 , further comprising communicating with the central file manager apparatus with a wide area network (WAN) optimization apparatus that operates as a client to file server nodes, wherein the apparatus implements server message block (SMB) signing on communications with the file server nodes including SMB signing on messages used to pre-fetch data from the file server nodes.
21. A non-transitory computer readable medium having stored thereon instructions for managing files comprising machine executable code which when executed by at least one processor, causes the processor to perform steps comprising:
storing an authoritative copy of one or more files selected from at least one of data files or metadata files;
communicating mirror copies of each authoritative copy to each of a plurality of remote network file managers;
obtaining updated file data from one of the plurality of remote network file managers when one of the mirror copies is updated by a client at the one remote network file manager;
updating the authoritative copy corresponding to the updated mirror copy with the updated file data;
notifying each of the other remote network file managers that the respective mirror copy stored thereon, and corresponding to the updated mirror copy, is no longer valid; and
asynchronously communicating a mirror copy of the updated authoritative copy to the other remote network file managers using a lazy mirroring technique based on available network bandwidth.
22. A medium according to claim 21 , wherein the central network file manager and the plurality of remote network file managers maintain a common global namespace.
23. A medium according to claim 22 further having stored thereon instructions that when executed by the processor causes the processor to perform steps further comprising communicating with the central file manager apparatus with a wide area network (WAN) optimization apparatus that operates as a client to file server nodes, wherein the apparatus implements server message block (SMB) signing on communications with the file server nodes including SMB signing on messages used to pre-fetch data from the file server nodes.
24. A medium according to claim 21 , wherein the communicating mirror copies of each authoritative copy further comprises asynchronously pushing each copy to each remote network file manager using a lazy mirroring technique based on available network bandwidth.
25. A medium according to claim 24 , further having stored thereon instructions that when executed by the processor causes the processor to perform steps further comprising:
after pushing each mirror copy to each remote network file manager, verifying that the data has not changed since being pushed; and
notifying each remote network file manager that the data is valid.
26. A medium according to claim 21 , wherein the communicating mirror copies of each authoritative copy further comprises asynchronous responding, using a lazy mirroring technique based on available network bandwidth, to a pull request for a mirror copy from one or more of the remote network file managers.
27. A medium according to claim 26 further having stored thereon instructions that when executed by the processor causes the processor to perform steps further comprising after communicating one or more mirror copies to one or more remote network file managers in response to a pull request, receiving a request, from each of the one or more remote network file managers, to verify that the data is still valid.
28. A medium according to claim 21 further having stored thereon instructions that when executed by the processor causes the processor to perform steps further comprising:
maintaining statistics with the central file manager apparatus regarding access patterns by remote clients; and
asynchronously communicating with the central file manager apparatus a mirror copy of the updated authoritative copy to the other remote network file managers based on the statistics.
29. A medium according to claim 21 , wherein the authoritative copies are mirrored in at least one of a breadth-first fashion or a depth-first fashion.
30. A medium according to claim 21 , wherein the communicating mirror copies of each authoritative copy further comprises asynchronously communicating each copy to each remote network file manager based on at least one of a first access to the authoritative copy by one or more of the remote network file managers, pre-fetching based on one or more authoritative copy access characteristic, and pre-loading of the authoritative copies prior to instantiation of one or more remote network file managers in the network at a remote site.