Systems and methods for implementing content sensitive routing over a wide area network (WAN)
Systems and methods for optimizing storage network functionality. The methods and systems of the present invention are particularly useful for optimizing storage network performance for cases in which some components of the network may be separated by significant distances and/or which include communication links with relatively limited bandwidth. In certain aspects, the present invention provides methods and systems for implementing access to and management of geographically distributed storage resources through multiple peer-to-peer storage network array management functions (AMFs) that may also be geographically distributed. The methods and systems of the present invention, in certain aspects, provide geographically aware cache sharing, cache replication, cache coherence, traffic routing, redundancy group structure, source and destination selection, pre-fetching of data, message gathering and other useful features.
1. A system for implementing hierarchical distributed cache coherence in a storage network, the system comprising:
a redundancy group (RG) including a first plurality of storage resources and a second plurality of storage resources located remotely from the first plurality of storage resources;
an array management function (AMF) access group (AAG), including a plurality of AMFs that share read and write access to the RG, wherein the AAG includes two or more local AMF access groups (LAAGs), each LAAG being either remote from other LAAGS or communicably coupled to other LAAGs by a low bandwidth network link; and
a proxy node configured to manage communications between the two or more LAAGs so as to minimize or eliminate inter-LAAG meta-data traffic for read and write operations;
wherein the AMF access group includes:
a set of first array management function devices forming a first LAAG which is constructed and arranged to access a redundancy group of first storage resources, and
a set of second array management function devices forming a second LAAG which is constructed and arranged to access a redundancy group of second storage resources, the first array management function devices being separate and distinct from the second array management function devices, and the first storage resources being separate and distinct from the second storage resources;
wherein a particular first array management function device is constructed and arranged to operate as a proxy node of the first LAAG;
wherein a particular second array management function device is constructed and arranged to operate as a proxy node of the second LAAG;
wherein the proxy node of the first LAAG is constructed and arranged to exchange cache coherency communications with the proxy node of the second LAAG; and
wherein the proxy node of the first LAAG has a local cache; and wherein the proxy node of the first LAAG is constructed and arranged to:
receive a write invalidate command signal from a first array management function device of the first LAAG,
invalidate particular blocks of the local cache of the proxy node of the first LAAG,
send a command message to the proxy node of the second LAAG directing the proxy node of the second LAAG to inform all of the second array management function devices to invalidate particular blocks of their local caches, the particular blocks of their local caches corresponding to the particular blocks of the local cache of the proxy node of the first LAAG; and
receive a reply message from the proxy node of the second LAAG indicating that the proxy node of the second LAAG has confirmed that all of the second array management function devices have invalidated the blocks of their local caches in response to the command message.
2. The system of claim 1 , wherein each proxy node is implemented in one of the plurality of AMFs or as a separate AMF.
3. The system of claim 1 , wherein each first array management function device is constructed and arranged to coordinate cache coherency with other first array management function devices in accordance with a local cache coherency protocol;
wherein each second array management function device is constructed and arranged to coordinate cache coherency with other second array management function devices in accordance with the local cache coherency protocol; and
wherein the proxy nodes of the first and second LAAGs are constructed and arranged to coordinate cache coherency between the first and second LAAGs in accordance with a remote cache coherency protocol which is different than the local cache coherency protocol.
4. The system of claim 3 , wherein bandwidth demand between two array management function devices in accordance with the local cache coherency protocol is higher than bandwidth demand between the proxy nodes of the first and second LAAGs in accordance with the remote cache coherency protocol.
5. The system of claim 1 , wherein the proxy node of the first LAAG is further constructed and arranged to:
send a response signal to the first array management function device of the first LAAG to report completion of block invalidation in response to the write invalidate command signal.
6. The system of claim 1 wherein the first LAAG has first storage control devices and first communications links connecting the first storage control devices together;
wherein the second LAAG has second storage control devices and second communications links connecting the second storage control devices together; and
wherein the low bandwidth network link connects a particular first storage control device of the first LAAG with a particular second storage control device of the second LAAG, each of the first and second communications links being constructed and arranged to provide a higher bandwidth than that of the low bandwidth network link which connects the particular first storage control device of the first LAAG with the particular second storage control device of the second LAAG.
7. The system of claim 6 wherein the first storage control devices of the first LAAG reside at a first location;
wherein the second storage control devices of the second LAAG reside at a second location, the first and second locations being geographically separated by at least a thousand kilometers;
wherein the particular first storage control device of the first LAAG is constructed and arranged to send write invalidation commands to the particular second storage control device of the second LAAG through the low bandwidth network link on behalf of the first LAAG in response to write operations received from a particular host which is locally connected to the first LAAG; and
wherein the particular second storage control device of the second LAAG is constructed and arranged to distribute the write invalidation commands from the particular first storage control device to other second storage control devices of the second LAAG through the second communications links.
8. The system of claim 7 wherein the proxy node is constructed and arranged to process inter-LAAG meta-data traffic for read and write operations between one of the first and second LAAGs and another LAAG.
9. The system of claim 1 wherein the proxy node of the first LAAG, when receiving the write invalidate command signal from the first array management function device of the first LAAG, is constructed and arranged to:
obtain the write invalidate command signal through a first communications medium connecting the proxy node of the first LAAG to the first array management function device of the first LAAG;
wherein the proxy node of the first LAAG, when sending the command message to the proxy node of the second LAAG, is constructed and arranged to:
transmit the command message to the proxy node of the second LAAG through a second communications medium connecting the proxy node of the first LAAG to the proxy node of the second LAAG; and
wherein the bandwidth of the second communications medium is substantially lower than the bandwidth of the first communications medium.
10. The system of claim 9 , wherein the second LAAG includes multiple AMF devices; and
wherein the command message transmitted to the proxy node of the second LAAG is constructed and arranged to direct the proxy node of the second LAAG to locally broadcast a write invalidate command to all of the multiple AMF devices of the second LAAG.
11. The system of claim 10 wherein the proxy node of the first LAAG is further constructed and arranged to:
in response to transmitting the command message to the proxy node of the second LAAG through the second communications medium, receive an acknowledgement of successful invalidation from the proxy node of the second LAAG through the second communications medium, the acknowledgement indicating that the proxy node of the second LAAG has received acknowledgement of successful write invalidations from all of the multiple AMF devices of the second LAAG in response to locally broadcasting the write invalidate command by the proxy node of the second LAAG.
12. A method for implementing hierarchical distributed cache coherence in a storage network, the network comprising a redundancy group (RG) including a first plurality of storage resources and a second plurality of storage resources located remotely from the first plurality of storage resources, and an array management function (AMF) access group (AAG), including a plurality of AMFs that share read and write access to the RG, wherein the AAG includes two or more local AMF access groups (LAAGs), each LAAG being either remote from other LAAGS or communicably coupled to other LAAGs by a low bandwidth network link, the method comprising:
receiving a data access request from a host by a first AMF in a first LAAG;
sending meta-data associated with the access request to a second AMF in the LAAG, wherein the second AMF is configured to act as a proxy agent for the LAAG; and
sending, from the second AMF, the meta-data to the proxy agent for each of the other LAAGs;
wherein the AMF access group includes:
a set of first array management function devices forming a first LAAG which is constructed and arranged to access a redundancy group of first storage resources, and
a set of second array management function devices forming a second LAAG which is constructed and arranged to access a redundancy group of second storage resources, the first array management function devices being separate and distinct from the second array management function devices, and the first storage resources being separate and distinct from the second storage resources;
wherein the method further comprises:
the first array management function devices forming the first LAAG performing read and write operations on the first storage resources, and
the second array management function devices forming the second LAAG performing read and write operations on the second storage resources;
wherein a particular first array management function device is constructed and arranged to operate as a proxy node of the first LAAG;
wherein a particular second array management function device is constructed and arranged to operate as a proxy node of the second LAAG;
wherein the method further comprises exchanging cache coherency communications between the proxy node of the first LAAG and the proxy node of the second LAAG;
wherein the proxy node of the first LAAG has a local cache; and
wherein the method further comprises:
receiving, by the proxy node of the first LAAG, a write invalidate command signal from a first array management function device of the first LAAG,
invalidating, by the proxy node of the first LAAG, particular blocks of the local cache of the proxy node of the first LAAG,
sending, by the proxy node of the first LAAG, a command message to the proxy node of the second LAAG directing the proxy node of the second LAAG to inform all of the second array management function devices to invalidate particular blocks of their local caches, the particular blocks of their local caches corresponding to the particular blocks of the local cache of the proxy node of the first LAAG, and
receiving, by the proxy node of the first LAAG, a reply message from the proxy node of the second LAAG indicating that the proxy node of the second LAAG has confirmed that all of the second array management function devices have invalidated the blocks of their local caches in response to the command message.
13. The method of claim 12 , wherein the data access request is a request to read data from a RG.
14. The method of claim 12 , wherein the data access request is a request to write data to a RG.
15. The method of claim 14 , wherein the meta-data includes a write invalidate command.
16. The method of claim 12 , wherein each first array management function device is constructed and arranged to coordinate cache coherency with other first array management function devices in accordance with a local cache coherency protocol;
wherein each second array management function device is constructed and arranged to coordinate cache coherency with other second array management function devices in accordance with the local cache coherency protocol; and
wherein the method further comprises coordinating, by the proxy nodes of the first and second LAAGs, cache coherency between the first and second LAAGs in accordance with a remote cache coherency protocol which is different than the local cache coherency protocol.
17. The method of claim 16 , wherein bandwidth demand between two array management function devices in accordance with the local cache coherency protocol is higher than bandwidth demand between the proxy nodes of the first and second LAAGs in accordance with the remote cache coherency protocol.
18. The method of claim 12 , further comprising:
sending, by the proxy node of the first LAAG, a response signal to the first array management function device of the first LAAG to report completion of block invalidation in response to the write invalidate command signal.
19. A storage system, comprising:
first storage resources arranged as a first redundancy group;
a set of first array management function devices forming a first local access group which is constructed and arranged to perform read and write operations on the first storage resources, a particular first array management function device being constructed and arranged to operate as a proxy node of the first local access group;
second storage resources arranged as a second redundancy group; and
a set of second array management function devices forming a second local access group which is constructed and arranged to perform read and write operations on the second storage resources, a particular second array management function device being constructed and arranged to operate as a proxy node of the second local access group;
the proxy node of the first local access group having a local cache and being constructed and arranged to: (i) receive a write invalidate command signal from a first array management function device of the first local access group, (ii) invalidate blocks of the local cache of the proxy node of the the first local access group, (iii) send a command message to the proxy node of the second local access group directing the proxy node of the second local access group to inform all of the second array management function devices to invalidate particular blocks of their local caches, the particular blocks of their local caches corresponding to the particular blocks of the local cache of the proxy node of the first local access group, and (iv) receive a reply message from the proxy node of the second LAAG indicating that the proxy node of the second LAAG has confirmed that all of the second array management function devices have invalidated the blocks of their local caches in response to the command message;
wherein first communications links connect the first array management function devices together;
wherein second communications links connect the second array management function devices together; and
wherein a low bandwidth network link connects the particular first array management function device to the particular second array management function device to provide communications between the first local access group and the second local access group, each of the first and second communications links being constructed and arranged to provide higher bandwidth than that of the low bandwidth network link, the first local access group and the second local access group being separated by at least a thousand kilometers.
20. A storage system as in claim 19 wherein the proxy node of the first local access group is further constructed and arranged to:
send a response signal to the first array management function device of the first LAAG to report completion of block invalidation in response to the write invalidate command signal.