PERSISTENT READ CACHE IN A SCALE OUT STORAGE SYSTEM
Methods, apparatuses, systems, and media for implementing a persistent read cache in a scale out storage system are disclosed to reduce access latency and achieve higher performance. Both the cached data blocks and distributed data placements are referenced by their unique content identifiers and are deduplicated. The persistent read cache spans across node reboots and is inherently coherent across all storage nodes without a distributed lock manager. The cached data blocks share the same storage pool as distributed data placements without costing storage capacity. A cached data block can become a distributed data placement or vice versa without moving the physical data block. Methods are also disclosed to reduce time to performance for logical device mobility.
1 . A method, comprising:
storing metadata mapping logical addresses associated with logical data blocks of one or more logical devices to physical addresses of physical data blocks stored in a plurality of data storage devices of a storage system, the metadata comprising:
first metadata mapping the logical addresses associated with the logical data blocks of the logical devices to a plurality of content identifiers;
a distributed hash table mapping the plurality of content identifiers to a plurality of node identifiers identifying a plurality of nodes of the storage system; and
second metadata mapping the content identifiers to the physical addresses of the physical data blocks;
creating distributed data placement based on the distributed hash table;
creating one or more cached data blocks, comprising:
caching the logical data blocks of each of the logical device on a node that provides an access point of the logical device; and
associating, by a processor, the cached data blocks with the content identifiers in the second metadata.
2 . The method of claim 1 , further comprising:
deduplicating the cached data blocks across the logical devices on one of the plurality of nodes based on the content identifiers associated with the cached data blocks.
3 . The method of claim 1 , further comprising:
removing one or more of the cached data blocks to allocate more space for the distributed data placement.
4 . The method of claim 3 , wherein removing the one or more of the cached data blocks comprises:
changing a flag in the second metadata indicative of whether a data block is a cached data block or a distributed data placement.
5 . The method of claim 1 , further comprising:
receiving, by a first processor on a first node of the plurality of nodes, a read request for a first logical block of a first logical device;
determining, based on a first content identifier associated with the first logical block, whether there is a local copy of the first logical block; and
reading, by the first processor, a physical data block based on the first content identifier in response to determining that there is a local copy of the first logical block.
6 . The method of claim 5 , further comprising:
in response to determining that there is no local copy of the first logical block, identifying a second node of the plurality of nodes using the distributed hash table; and
sending, to a second processor on the second node, a request for reading the physical data block.
7 . The method of claim 1 , further comprising:
in view of migration of an access point of a logical device from a third node to a fourth node of the plurality of nodes, identifying one or more cached data blocks on the third node that are associated with the logical device; and
pushing one or more of the identified cached data blocks to the fourth node.
8 . The method of claim 7 , wherein pushing the one or more of the identified cached data blocks to the fourth node comprises:
selecting the one or more of the identified cached data blocks based on access frequency and recency; and
pushing selected cached data blocks to the fourth node.
9 . A system, comprising:
a memory; and
a processor operatively coupled to the memory, the processor to:
store metadata mapping logical addresses associated with logical data blocks of one or more logical devices to physical addresses of physical data blocks stored in a plurality of data storage devices of a storage system, the metadata comprising:
first metadata mapping the logical addresses associated with the logical data blocks of the logical devices to a plurality of content identifiers;
a distributed hash table mapping the plurality of content identifiers to a plurality of node identifiers identifying a plurality of nodes of the storage system; and
second metadata mapping the content identifiers to the physical addresses of the physical data blocks
create distributed data placement based on the distributed hash table;
create one or more cached data blocks by caching the data blocks of each of the logical device on a node that provides an access point of the logical device; and
associate, by the processor, the cached data blocks with the content identifiers in the second metadata.
10 . The system of claim 9 , wherein the processor is further to:
deduplicate the cached data blocks across the logical devices on one of the plurality of nodes based on the content identifiers associated with the cached data blocks.
11 . The system of claim 9 , wherein the processor is further to:
remove one or more of the cached data blocks to allocate more space for the distributed data placement.
12 . The system of claim 11 , wherein, to remove the one or more of the cached data blocks, the processor is further to:
change a flag in the second metadata indicative of whether a data block is a cached data block or a distributed data placement.
13 . The system of claim 9 , wherein the processor is further to:
receive a read request for a first logical block of a first logical device;
determine, based on a first content identifier associated with the first logical block, whether there is a local copy of the first logical block; and
read a physical data block based on the first content identifier in response to determining that there is a local copy of the first logical block.
14 . The system of claim 13 , wherein the processor is further to:
in response to determining that there is no local copy of the first logical block, identify a second node of the plurality of nodes using the distributed hash table; and
send, to a processor on the second node, a request for reading the physical block.
15 . The system of claim 9 , wherein the processor is further to:
in view of moving of an access point of a logical device from a third node to a fourth node of the plurality of nodes, identify one or more cached data blocks on the third node that are associated with the logical device; and
push one or more of the identified cached data blocks to the fourth node.
16 . The system of claim 15 , wherein, to push the one or more of the identified cached data blocks to the fourth node, the processor is further to:
select the one or more of the identified cached data blocks based on access frequency and recency; and
push selected cached data blocks to the fourth node.
17 . A non-transitory machine-readable storage medium including instructions that, when accessed by a processor, cause the processor to:
store metadata mapping logical addresses associated with logical data blocks of one or more logical devices to physical addresses of physical data blocks stored in a plurality of data storage devices of a storage system, the metadata comprising:
first metadata mapping the logical addresses associated with the logical data blocks of the logical devices to a plurality of content identifiers;
a distributed hash table mapping the plurality of content identifiers to a plurality of node identifiers identifying a plurality of nodes of the storage system; and
second metadata mapping the content identifiers to the physical addresses of the physical data blocks
create distributed data placement based on the distributed hash table;
create one or more cached data blocks by caching the data blocks of each of the logical device on a node that provides an access point of the logical device; and
associate, by the processor, the cached data blocks with the content identifiers in the second metadata.
18 . The non-transitory machine-readable storage medium of claim 17 , wherein the processor is further to:
deduplicate the cached data blocks across the logical devices on one of the plurality of nodes based on the content identifiers associated with the cached data blocks.
19 . The non-transitory machine-readable storage medium of claim 18 , wherein the processor is further to:
remove one or more of the cached data blocks to allocate more space for the distributed data placement.
20 . The non-transitory machine-readable storage medium of claim 17 , wherein the processor is further to:
receive a read request for a first logical block of a first logical device;
determine, based on a first content identifier associated with the first logical block, whether there is a local copy of the first logical block; and
read a physical data block based on the first content identifier in response to determining that there is a local copy of the first logical block.