Scalable metadata acceleration with datapath metadata backup
View Patent ↗A system, method and computer program product for scalable metadata acceleration with datapath metadata backup as disclosed includes providing a plurality of metadata comprising variable block pointers for the physical location of system data. The method also includes storing at least a first and a second metadata in two or more solid state memories accelerated with respect to a read and a write of a memory in a datapath, each solid state memory configured as a logical copy of the other. The method additionally includes interleaving a backup metadata in a datapath memory at a variable interval based on a table of block pointers. The method further includes querying the accelerated metadata for an accelerated datapath operation of the system and querying the interleaved metadata and/or a logical mirror of the accelerated metadata for a rebuild operation of the accelerated metadata, each query determinable by the table of block pointers.
1. A method for managing metadata of a storage system, comprising:
a) providing a plurality of metadata comprising block pointers for physical locations of system data and deduplication, backup, compression, redundancy, error correction, rebuild, access and encryption data;
b) storing at least a first and a second metadata of the metadata plurality in two or more solid state memories accelerated with respect to a read and a write of a memory in a datapath, each solid state memory configured as a logical copy of the other;
c) interleaving a backup metadata in a datapath memory, the datapath configured to read and write the backup metadata in the datapath memory at a variable interval based on a table of block pointers; and
d) querying accelerated metadata for an accelerated datapath operation of the system and querying the interleaved metadata for a rebuild operation of the accelerated metadata, each query determinable by the table of block pointers.
2. The method for managing metadata of a storage system of claim 1 , wherein the plurality of metadata further comprises information regarding error correction schemes, data compression algorithms, RAID (redundant array of independent disks/devices) level implemented on a data block and whether a block belongs to one of a file and a LUN (logical unit number) with an offset address for SAN (storage area network) and NAS (network attached storage).
3. The method for managing metadata of a storage system of claim 1 , wherein interleaving the backup metadata in the datapath memory further comprises configuring the datapath to read and write the backup metadata in the datapath memory at a variable interval based on a table of variable block pointers and an interleaved metadata variable offset to a respective block pointer.
4. The method for managing metadata of a storage system of claim 3 , wherein the metadata interleaved in the datapath is interleaved at a plurality of variable intervals determined by the table of variable block pointers and the interleaved metadata variable offset to the respective block pointer.
5. The method for managing metadata of a storage system of claim 1 , wherein each solid state memory is configured in any logical mirror arrangement and any RAID arrangement.
6. The method for managing metadata of a storage system of claim 1 , wherein the two or more solid state memories may comprise a non-volatile semiconductor memory, a ferroelectric RAM (random access memory), a magneto-resistive RAM, a DRAM (dynamic random access memory), an SRAM (static random access memory) memory and any combination of solid state memory.
7. The method for managing metadata of a storage system of claim 1 , further comprising performing an inline data compression/decompression and depuplication of the accelerated metadata during a read of the datapath and a write of the datapath.
8. The method for managing metadata of a storage system of claim 1 , further comprising updating the accelerated metadata and updating the backup metadata concurrently and thereby accelerating a write of the interleaved backup metadata.
9. The method for managing metadata of a storage system of claim 1 , wherein a rebuild operation of the accelerated metadata comprises reading the interleaved backup metadata from the datapath memory per the table of block pointers into the accelerated metadata.
10. The method for managing metadata of a storage system of claim 1 , wherein a rebuild operation of the accelerated metadata further comprises rebuilding failed accelerated metadata in one solid state memory from one of the solid state memories and the interleaved backup metadata per the table of block pointers.
11. The method for managing metadata of a storage system of claim 1 , further comprising querying at least one of the accelerated solid state memories for a rebuild operation of a failed interleaved metadata.
12. The method for managing metadata of a storage system of claim 1 , further comprising scaling the solid state memories to include an additional pair multiple in a RAID 1+0 configuration to accommodate a scalable datapath memory including a JBOD (just a bunch of disks) chassis slot expansion.
13. A computer program product comprising a non-transitory computer readable storage medium having computer useable instruction codes executable to perform operations for managing metadata of a storage system, the operations of the computer program product comprising:
a) providing a plurality of metadata comprising block pointers for physical locations of system data and deduplication, compression, redundancy, error correction, rebuild, access and encryption data;
b) storing at least a first and a second metadata of the metadata plurality in two or more solid state memories accelerated with respect to a read and a write of a memory in a datapath, each solid state memory configured as a logical copy of the other;
c) interleaving a backup metadata in a datapath memory, the datapath memory configured to read and write the backup metadata in the datapath memory at a variable interval based on a table of block pointers; and
d) querying accelerated metadata for an accelerated datapath operation of the system and querying the interleaved metadata for a rebuild operation of the accelerated metadata, each query determinable by the table of block pointers.
14. The computer program product of claim 13 , wherein interleaving the backup metadata in the datapath memory further comprises configuring the datapath to read and write the backup metadata in the datapath memory at a variable interval based on a table of variable block pointers and an interleaved metadata variable offset to a respective block pointer.
15. The computer program product of claim 13 , wherein a rebuild operation of the accelerated metadata further comprises rebuilding failed accelerated metadata in one solid state memory from one of the solid state memories and the interleaved backup metadata per the table of block pointers.
16. The computer program product of claim 13 , further comprising scaling the solid state memories to include an additional pair multiple in a RAID 1+0 configuration to accommodate a scalable datapath memory including a JBOD (just a bunch of disks) chassis slot expansion.
17. A storage system with metadata management, comprising:
a) a plurality of metadata comprising block pointers for physical locations of system data and deduplication, snapshot saves, compression, redundancy, error correction, rebuild, access and encryption data;
b) at least a first and a second metadata of the metadata plurality in two or more solid state memories of the storage system accelerated with respect to a read and a write of a memory in a datapath, each solid state memory configured as a logical copy of the other;
c) a backup metadata interleaved in a datapath memory, the datapath configured to read and write the backup metadata in the datapath at a variable interval based on a table of block pointers; and
d) a query logic configured to accelerate metadata for an accelerated datapath operation of the system and query the interleaved metadata for a rebuild operation of the accelerated metadata, each query determinable by the table of block pointers.
18. The storage system with metadata management of claim 17 , wherein the plurality of metadata further comprises data regarding error correction schemes, data compression algorithms, RAID (redundant array of independent disks/devices) level implemented on a data block and whether a block belongs to one of a file and a LUN (logical unit number) with an offset address for SAN (storage area network) and NAS (network attached storage).
19. The storage system with metadata management of claim 17 , wherein the datapath comprises an accelerated memory for a read and a write of data between a source memory and a destination memory, the accelerated memory accelerated with respect to a read and a write of a mechanical disk memory in a storage area network (SAN).
20. The storage system with metadata management of claim 17 , further comprising scaling the solid state memories to include an additional pair multiple in a RAID 1+0 configuration to accommodate a scalable datapath memory including a JBOD (just a bunch of disks) chassis slot expansion.