Virtualized data storage system architecture
View Patent ↗Virtual storage arrays consolidate branch data storage at data centers connected via wide area networks. Virtual storage arrays appear to storage clients as local data storage; however, virtual storage arrays actually store data at the data center. The virtual storage arrays overcomes bandwidth and latency limitations of the wide area network by predicting and prefetching storage blocks, which are then cached at the branch location. Virtual storage arrays leverage an understanding of the semantics and structure of high-level data structures associated with storage blocks to predict which storage blocks are likely to be requested by a storage client in the near future. Virtual storage arrays determine the association between requested storage blocks and corresponding high-level data structure entities to predict additional high-level data structure entities that are likely to be accessed. From this, the virtual storage array identifies the additional storage blocks for prefetching.
1. A method, comprising:
in response to receiving a first read request for a first storage block, retrieving the first storage block from a cache or a storage device;
identifying a first high level data structure associated with the first storage block, wherein the first high level data structure is one of: a first file, a first directory, a first file system node, a first database, or a first row;
analyzing the first high level data structure to determine a second high level data structure that (1) is referenced in the first high level data structure, and (2) is likely to be associated with a future read request, wherein the second high level data structure is one of: a second file, a second directory, a second file system node, a second database, or a second row;
identifying a second storage block associated with the second high level data structure, wherein said identifying the second storage block comprises:
providing a first identifier of the second high-level data structure to an inferred storage structure database, and
receiving a second identifier of the second storage block from the inferred storage structure database;
in response to determining, by using a storage block access optimizer, that the second storage block is not present in the cache,
prefetching the second storage block from the storage device, and
storing the second storage block in the cache; and
in response to receiving a second read request for the second storage block, retrieving the second storage block from the cache.
2. The method of claim 1 , wherein the cache is in a first network location, wherein the storage device is in a second network location, and wherein the first network location is connected to the second network location by a wide area network.
3. The method of claim 1 , further comprising:
receiving, from a client, a write request to write an updated third storage block to the storage device;
in response to determining that the cache is full,
sending the updated third storage block to the storage device,
receiving a first write acknowledgement from the storage device, and
sending the first write acknowledgement to the client; and
in response to determining that the cache is not full,
storing the updated third storage block in the cache,
sending a second write acknowledgement to the client, and
sending the updated third storage block to the storage device.
4. The method of claim 1 , further comprising:
over a period of time that includes a set of snapshot times, receiving multiple write requests to write versions of a third storage block to the storage device; and
maintaining a latest version of the third storage block in the cache that was written to the cache before each snapshot time in the set of snapshot times.
5. The method of claim 4 , further comprising:
sending, to the storage device, the latest version of the third storage block in the cache that was written to the cache before each snapshot time in the set of snapshot times.
6. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform a method, comprising:
in response to receiving a first read request for a first storage block, retrieving the first storage block from a cache or a storage device;
identifying a first high level data structure associated with the first storage block, wherein the first high level data structure is one of: a first file, a first directory, a first file system node, a first database, or a first row;
analyzing the first high level data structure to determine a second high level data structure that (1) is referenced in the first high level data structure, and (2) is likely to be associated with a future read request, wherein the second high level data structure is one of: a second file, a second directory, a second file system node, a second database, or a second row;
identifying a second storage block associated with the second high level data structure, wherein said identifying the second storage block comprises:
providing a first identifier of the second high-level data structure to an inferred storage structure database, and
receiving a second identifier of the second storage block from the inferred storage structure database;
in response to determining, by using a storage block access optimizer, that the second storage block is not present in the cache,
prefetching the second storage block from the storage device, and storing the second storage block in the cache; and
in response to receiving a second read request for the second storage block, retrieving the second storage block from the cache.
7. The non-transitory computer-readable storage medium of claim 6 , wherein the cache is in a first network location, wherein the storage device is in a second network location, and wherein the first network location is connected to the second network location by a wide area network.
8. The non-transitory computer-readable storage medium of claim 6 , wherein the method further comprising:
receiving, from a client, a write request to write an updated third storage block to the storage device;
in response to determining that the cache is full,
sending the updated third storage block to the storage device,
receiving a first write acknowledgement from the storage device, and
sending the first write acknowledgement to the client; and in response to determining that the cache is not full,
storing the updated third storage block in the cache,
sending a second write acknowledgement to the client, and
sending the updated third storage block to the storage device.
9. The non-transitory computer-readable storage medium of claim 6 , wherein the method further comprising:
over a period of time that includes a set of snapshot times, receiving multiple write requests to write versions of a third storage block to the storage device; and
maintaining a latest version of the third storage block in the cache that was written to the cache before each snapshot time in the set of snapshot times.
10. The non-transitory computer-readable storage medium of claim 9 , wherein the method further comprising:
sending, to the storage device, the latest version of the third storage block in the cache that was written to the cache before each snapshot time in the set of snapshot times.
11. An apparatus, comprising:
a processor; and
a non-transitory computer-readable storage medium storing instructions that, when executed by the processor, cause the processor to perform a method, comprising: in response to receiving a first read request for a first storage block, retrieving the first storage block from a cache or a storage device;
identifying a first high level data structure associated with the first storage block, wherein the first high level data structure is one of: a first file, a first directory, a first file system node, a first database, or a first row;
analyzing the first high level data structure to determine a second high level data structure that (1) is referenced in the first high level data structure, and (2) is likely to be associated with a future read request, wherein the second high level data structure is one of: a second file, a second directory, a second file system node, a second database, or a second row;
identifying a second storage block associated with the second high level data structure, wherein said identifying the second storage block comprises:
providing a first identifier of the second high-level data structure to an inferred storage structure database, and
receiving a second identifier of the second storage block from the inferred storage structure database;
in response to determining, by using a storage block access optimizer, that the second storage block is not present in the cache,
prefetching the second storage block from the storage device, and
storing the second storage block in the cache; and
in response to receiving a second read request for the second storage block, retrieving the second storage block from the cache.
12. The apparatus of claim 11 , wherein the cache is in a first network location, wherein the storage device is in a second network location, and wherein the first network location is connected to the second network location by a wide area network.
13. The apparatus of claim 11 , wherein the method further comprising:
receiving, from a client, a write request to write an updated third storage block to the storage device;
in response to determining that the cache is full,
sending the updated third storage block to the storage device,
receiving a first write acknowledgement from the storage device, and
sending the first write acknowledgement to the client; and
in response to determining that the cache is not full,
storing the updated third storage block in the cache,
sending a second write acknowledgement to the client, and
sending the updated third storage block to the storage device.
14. The apparatus of claim 11 , wherein the method further comprising:
over a period of time that includes a set of snapshot times, receiving multiple write requests to write versions of a third storage block to the storage device; and
maintaining a latest version of the third storage block in the cache that was written to the cache before each snapshot time in the set of snapshot times.
15. The apparatus of claim 14 , wherein the method further comprising:
sending, to the storage device, the latest version of the third storage block in the cache that was written to the cache before each snapshot time in the set of snapshot times.