VIRTUALIZED DATA STORAGE SYSTEM ARCHITECTURE USING PREFETCHING AGENT
Virtual storage arrays consolidate data storage from branch locations at data centers. The virtual storage array appears to storage clients as a local data storage; however, the virtual storage array data is actually stored at a data center. To overcome the bandwidth and latency limitations of wide area networks between branch locations and the data center, systems and methods predict, prefetch, and cache at the branch location storage blocks that are likely to be requested in the future by storage clients. When this prediction is successful, storage block requests are fulfilled from branch locations' storage block caches. Predictions may leverage an understanding of the semantics and structure of the high-level data structures associated with the storage blocks. Prefetching agents on storage clients monitor storage requests to determine the associations between requested storage blocks and the corresponding high-level data structures as well as other attributes useful for prediction.
1 . A method of optimizing a block storage protocol access to a block storage device, the method comprising:
receiving a first storage request for access to a first storage block from a storage client;
receiving storage block prefetching information from a prefetching agent monitoring the storage client;
using the storage block prefetching information, identifying a first high-level data structure entity corresponding with the first storage block;
analyzing the first high-level data structure entity to identify at least at a portion of a set of high-level data structure entities likely to be associated with at least one future storage request;
identifying a second storage block corresponding with the portion of the set of high-level data structure entities; and
communicating the second storage block from a physical storage array to a storage block cache via a wide-area network.
2 . The method of claim 1 , wherein analyzing the first high-level data structure entity comprises:
using the storage block prefetching information in addition to the first high-level data structure entity to identify the portion of a set of high-level data structure entities.
3 . The method of claim 2 , wherein the storage block prefetching information includes an identity of a storage client application associated with the first storage request.
4 . The method of claim 1 , wherein the prefetching agent is operating on the storage client.
5 . The method of claim 4 , wherein the prefetching agent is included in an application or a background process operating on the storage client.
6 . The method of claim 4 , wherein the prefetching agent is included in a driver module operating on the storage client.
7 . The method of claim 1 , wherein the storage client is operating within a virtual machine and the prefetching agent is included in a virtualization platform including the virtual machine.
8 . The method of claim 1 , wherein receiving the storage block prefetching information from the prefetching agent comprises:
monitoring a predetermined control file to detect the storage of the storage block prefetching information in the predetermined control file.
9 . The method of claim 8 , wherein the predetermined control file and the first high-level data structure entity are included in the same virtual storage array.
10 . The method of claim 8 , wherein the predetermined control file and the first high-level data structure entity are included in different virtual storage arrays.
11 . The method of claim 1 , wherein the first high-level data structure entity is included in a first virtual storage array and receiving the storage block prefetching information from the prefetching agent comprises:
monitoring a second virtual storage array for second storage requests from the prefetching agent.
12 . The method of claim 11 , wherein the second storage requests include a write operation to a copy of the first high-level data structure entity included in the second virtual storage array.
13 . The method of claim 1 , wherein receiving the storage block prefetching information from the prefetching agent comprises:
receiving the storage block prefetching information via a network connection with the prefetching agent.
14 . The method of claim 1 , wherein receiving the storage block prefetching information from the prefetching agent comprises:
receiving the storage block prefetching information via a network connection with the prefetching agent.
15 . The method of claim 1 , wherein receiving the storage block prefetching information from the prefetching agent comprises:
receiving the storage block prefetching information via a storage block protocol.
16 . The method of claim 15 , wherein the storage block prefetching information is included in metadata added to storage block access requests via the storage block protocol.
17 . The method of claim 16 , wherein the metadata is included in the first storage request.
18 . The method of claim 1 , wherein the storage block cache is included in a local network including the storage client.
19 . The method of claim 1 , wherein the storage block cache is included in the storage client.
20 . The method of claim 1 , wherein the first high-level data structure entity includes a file.