Virtualized Data Storage Over Wide-Area Networks
Virtual storage arrays consolidate branch data storage at data centers connected via wide area networks. Virtual storage arrays appear to storage clients as local data storage; however, virtual storage arrays actually store data at the data center. The virtual storage arrays overcomes bandwidth and latency limitations of the wide area network by predicting and prefetching storage blocks, which are then cached at the branch location. Virtual storage arrays leverage an understanding of the semantics and structure of high-level data structures associated with storage blocks to predict which storage blocks are likely to be requested by a storage client in the near future. Virtual storage arrays determine the association between requested storage blocks and corresponding high-level data structure entities to predict additional high-level data structure entities that are likely to be accessed. From this, the virtual storage array identifies the additional storage blocks for prefetching.
1 . A method of optimizing a block storage protocol read access to a block storage device via a wide area network, the method comprising:
receiving a storage request specifying at least a first storage block from a storage client, wherein the storage client is connected with a wide area network at a first network location;
identifying at least a first portion of a set of file system entities corresponding with the first storage block;
identifying at least at a second portion of the set of file system entities likely to be associated with a future storage request based on the first portion of the set of file system entities;
identifying at least a second storage block corresponding with the second portion of the set of file system entities;
retrieving the second storage block from a data storage connected with the wide area network at a second network location;
communicating via the wide area network the second storage block from the data storage to a storage block cache at the first network location; and
storing the second storage block in the storage block cache.
2 . The method of claim 1 , wherein the first portion of the set of file system entities and the second portion of the set of file system entities include a first one of the set of file system entities.
3 . The method of claim 1 , wherein the first portion of the set of file system entities includes a first one of the set of file system entities and the second portion of the set of file system entities includes a second one of the set of file system entities.
4 . The method of claim 1 , wherein the set of file system entities includes a file system entity.
5 . The method of claim 1 , wherein the set of file system entities includes a directory.
6 . The method of claim 1 , wherein the set of file system entities includes a file system data structure.
7 . The method of claim 1 , wherein identifying at least the first portion of a set of file system entities corresponding with the first storage block comprises:
accessing a storage structure database including mappings from storage block locations to portions of the set of file system entities.
8 . The method of claim 1 , wherein identifying at least the second storage block corresponding with the second portion of the set of file system entities comprises:
accessing a data storage structure including previously determined mappings from portions of the set of file system entities to storage block locations.
9 . The method of claim 1 , comprising:
receiving a second storage request from the storage client;
determining if the second storage request includes a request for the second storage block;
in response to the determination that the second storage request includes the request for the second storage block, retrieving the second storage block from the storage block cache at the first network location; and
in response to the determination that the second storage request does not include the request for the second storage block, retrieving at least one additional storage block from the data storage connected with the wide area network at the second network location.
10 . A method of optimizing a block storage protocol read access to a block storage device via a wide area network, the method comprising:
receiving a storage request specifying at least a first storage block from a storage client, wherein the storage client is connected with a wide area network at a first network location;
identifying at least a first portion of a set of database entities corresponding with the first storage block;
identifying at least at a second portion of the set of database entities likely to be associated with a future storage request based on the first portion of the set of database entities;
identifying at least a second storage block corresponding with the second portion of the set of database entities;
retrieving the second storage block from a data storage connected with the wide area network at a second network location;
communicating via the wide area network the second storage block from the data storage to a storage block cache at the first network location; and
storing the second storage block in the storage block cache.
11 . The method of claim 10 , wherein the first portion of the set of database entities and the second portion of the set of database entities include a first one of the set of database entities.
12 . The method of claim 10 , wherein the first portion of the set of database entities includes a first one of the set of database entities and the second portion of the set of database entities includes a second one of the set of database entities.
13 . The method of claim 10 , wherein the set of database entities includes a table.
14 . The method of claim 10 , wherein the set of database entities includes a database system node.
15 . The method of claim 10 , wherein identifying at least the first portion of a set of database entities corresponding with the first storage block comprises:
accessing a storage structure database including mappings from storage block locations to portions of the set of database entities.
16 . The method of claim 10 , wherein identifying at least the second storage block corresponding with the second portion of the set of database entities comprises:
accessing a data storage structure including previously determined mappings from portions of the set of database entities to storage block locations.
17 . The method of claim 10 , comprising:
receiving a second storage request from the storage client;
determining if the second storage request includes a request for the second storage block;
in response to the determination that the second storage request includes the request for the second storage block, retrieving the second storage block from the storage block cache at the first network location; and
in response to the determination that the second storage request does not include the request for the second storage block, retrieving at least one additional storage block from the data storage connected with the wide area network at the second network location.
18 . A method of optimizing a block storage protocol write access to a block storage device via a wide area network, the method comprising:
receiving a storage request specifying at least a first storage block from a storage client connected with a wide area network at a first network location;
determining if a storage block cache has sufficient capacity to store at least the first storage block; and
in response to the determination that the storage block cache has sufficient capacity to store at least the first storage block:
storing the first storage block in the storage block cache;
sending a storage request acknowledgement to the storage client indicating that the storage request is complete; and
following the storage request acknowledgement, communicating the first storage block via the wide area network to a data storage connected with the wide area network at a second network location, wherein the data storage is adapted to store the first storage block.
19 . The method of claim 18 , wherein the storage block cache is located at the first network location and is connected with the storage client via a first local network.
20 . The method of claim 18 , further comprising:
in response to the determination that the storage block cache does not have sufficient capacity to store at least the first storage block:
communicating the first storage block via the wide area network to a data storage connected with the wide area network at a second network location;
receiving a first storage request acknowledgement from the data storage, wherein the first storage request acknowledgment indicates that the data center has stored the first storage block; and
following the receipt of the first storage request acknowledgement, sending a second storage request acknowledgement to the storage client indicating that the storage request is complete.
21 . A method of preserving data in a data storage device, the method comprising:
setting a storage interface connected with a wide area network at a first network location to a quiescent state;
identifying a first set of storage blocks in a storage block cache connected at the first network location that has changed since following its initial storage in the storage block cache;
setting the storage interface to an active state;
following the storage interface setting to the active state, transferring the first set of storage blocks via the wide area network to a second network location; and
storing a data snapshot on a data storage at the second network location, wherein the snapshot includes the first set of storage blocks.
22 . The method of claim 21 , wherein transferring is in response to a snapshot request received from an administration application.
23 . The method of claim 21 , wherein the data snapshot includes a copy of a second set of storage blocks stored by the data storage, wherein the second set of storage blocks is unchanged since the time of that the storage interface is set to the quiescent state.
24 . The method of claim 23 , wherein the second set of storage blocks was previously stored by the data storage prior to the storage interface being set to the quiescent state.
25 . The method of claim 21 , comprising:
receiving, prior to transferring the first set of storage blocks, a first modification to at least a portion of the first set of storage blocks;
in response to receiving the first modification, creating a copy of at least the portion of the first set of storage blocks;
applying the first modification to the copy of at least the portion of the first set of storage blocks; and
preserving the unmodified portion of the first set of storage blocks for transfer to the second network location.
26 . The method of claim 25 , comprising:
receiving a storage request from a storage client at the first network location, wherein the storage request specifies at least the portion of the first set of storage blocks;
in response to the storage request, providing the modified copy of the portion of the first set of storage blocks to the storage client.