Technique for improving operations log indexing
A technique improves implementation of an index for an operations log (oplog) that coalesces random write operations directed to a virtual disk (vdisk) stored on an extent store. The oplog temporarily caches data associated with the random write operations (i.e., write data) as well as metadata describing the write data. The metadata includes descriptors to the write data stored on virtual address regions, i.e., offset ranges, of the vdisk and are used to identify the offset ranges of write data for the vdisk that are cached in the oplog. To facilitate fast lookup operations of the offset ranges when determining whether write data is cached in the oplog, an oplog index provides a state of the latest data for offset ranges of the vdisk. The technique improves implementation of the oplog index by storing the oplog index in storage class memory, such as persistent memory, to obviate failure and subsequent recovery of the oplog index.
1. A non-transitory computer readable medium including program instructions for execution on a processor, the program instructions configured to:
record, in a persistent memory (PMEM) of a node of a cluster, metadata corresponding to input/output (I/O) write accesses having data directed to a virtual disk (vdisk) of the cluster, the metadata locating the data cached in an operations log (oplog) prior to persistent storage of the data at a backend of the cluster;
index the metadata by offset ranges of the vdisk;
store a first probabilistic data structure created from the metadata in a dynamic random access memory (DRAM) of the node, wherein the first probabilistic data structure includes locations of the data cached in the oplog and drained from the oplog;
store in the DRAM, a second probabilistic data structure having the metadata locating the data cached in the oplog;
in response to an I/O access having an offset directed to the vdisk, use the second probabilistic data structure and then the first probabilistic data structure to determine whether the offset is in the metadata; and
in response to determining that the offset is in the metadata, search the metadata using the offset to locate and retrieve the data.
2. The non-transitory computer readable medium of claim 1 , wherein the first and second probabilistic data structures are Bloom filters.
3. The non-transitory computer readable medium of claim 1 , wherein the program instructions are further configured to:
in response to a reduction in a total capacity of the DRAM and PMEM, drain the data cached in the oplog to the backend of the cluster to increase a capacity to store the metadata.
4. The non-transitory computer readable medium of claim 1 , wherein the program instructions are further configured to:
in response to the locations of the data in the oplog included in the first probabilistic data structure being drained from the oplog, mark the first probabilistic data structure as frozen for updates and create the second probabilistic data structure.
5. The non-transitory computer readable medium of claim 4 , wherein the second probabilistic data structure is updated for locations of new data directed to the vdisk and cached in the oplog.
6. The non-transitory computer readable medium of claim 1 , wherein the first probabilistic data structure is deleted once the data in the oplog having the locations in the first probabilistic data structure is drained from the oplog.
7. A method comprising:
recording, in a storage class memory of a node of a cluster, metadata corresponding to input/output (I/O) write accesses having data directed to a virtual disk (vdisk) of the cluster, the metadata locating the data cached in an operations log (oplog) prior to persistent storage of the data at a backend of the cluster;
indexing the metadata by offset ranges of the vdisk;
storing a first Bloom filter created from the metadata in a dynamic random access memory (DRAM) of the node, wherein the first Bloom filter includes locations of the data cached in the oplog and drained from the oplog;
storing in the DRAM, a second Bloom filter having the metadata locating the data cached in the oplog;
in response to an I/O access having an offset directed to the vdisk, using the second Bloom filter and then the first Bloom filter to determine whether the offset is in the metadata; and
in response to determining that the offset is in the metadata, searching the metadata to retrieve the offset to locate the data, wherein the second Bloom filter is deleted once the data in the oplog having the locations in the second Bloom filter is drained from the oplog.
8. A method comprising:
recording, in a persistent memory (PMEM) of a node of a cluster, metadata corresponding to input/output (I/O) write accesses having data directed to a virtual disk (vdisk) of the cluster, the metadata locating the data cached in an operations log (oplog) prior to persistent storage of the data at a backend of the cluster;
indexing the metadata by offset ranges of the vdisk;
storing a first probabilistic data structure created from the metadata in a dynamic random access memory (DRAM) of the node, wherein the first probabilistic data structure includes locations of the data cached in the oplog and drained from the oplog;
storing in the DRAM, a second probabilistic data structure having the metadata locating the data cached in the oplog;
in response to an I/O access having an offset directed to the vdisk, using the second probabilistic data structure and then the first probabilistic data structure to determine whether the offset is in the metadata; and
in response to determining that the offset is in the metadata, searching the metadata using the offset to locate and retrieve the data.
9. The method of claim 8 , wherein the first and second probabilistic data structures are Bloom filters.
10. The method claim 8 further comprising:
in response to a reduction in a total capacity of the DRAM and PMEM, draining the data cached in the oplog to the backend of the cluster to increase a capacity to store the metadata.
11. The method of claim 8 further comprising:
in response to the locations of data in the oplog included in the first probabilistic data structure being drained from the oplog, marking the first probabilistic data structure as frozen for updates and create the second probabilistic data structure.
12. The method of claim 8 , wherein the second probabilistic data structure is updated for locations of new data directed to the vdisk and cached in the oplog.
13. The method of claim 8 , wherein the first probabilistic data structure is deleted once the data in the oplog having the locations in the first probabilistic data structure is drained from the oplog.
14. An apparatus comprising:
a cluster of nodes each having a processor and storage; and
a network interconnecting the nodes and connecting to a client, wherein the processors of the nodes are configured to:
record, in a persistent memory (PMEM) of a node of the cluster, metadata corresponding to input/output (I/O) write accesses having data from the client directed to a virtual disk (vdisk) of the cluster, the metadata locating the data cached in an operations log (oplog) prior to persistent storage of the data at a backend of the cluster;
index the metadata by offset ranges of the vdisk;
store a first probabilistic data structure created from the metadata in a dynamic random access memory (DRAM) of the node, wherein the first probabilistic data structure includes locations of the data cached in the oplog and drained from the oplog;
store in the DRAM, a second probabilistic data structure having the metadata locating the data cached in the oplog;
in response to an I/O access having an offset directed to the vdisk, use the second probabilistic data structure and then the first probabilistic data structure to determine whether the offset is in the metadata; and
in response to determining that the offset is in the metadata, search the metadata using the offset to locate and retrieve the data.
15. The apparatus of claim 14 , wherein the first and second probabilistic data structures are Bloom filters.
16. The apparatus of claim 14 wherein the processors of the nodes are further configured to:
in response to a reduction in a total capacity of the DRAM and PMEM, drain the data cached in the oplog to the backend of the cluster to increase a capacity to store the metadata.
17. The apparatus of claim 14 wherein the processors of the nodes are further configured to:
in response to the locations of data in the oplog included in the first probabilistic data structure being drained from the oplog, mark the first probabilistic data structure as frozen for updates and create the second probabilistic data structure.
18. The apparatus of claim 14 , wherein the second probabilistic data structure is updated for locations of new data directed to the vdisk and cached in the oplog.
19. The apparatus of claim 14 , wherein the first probabilistic data structure is deleted once the data in the oplog having the locations in the first probabilistic data structure is drained from the oplog.