Distributed generation of random data in a storage system
View Patent ↗A storage system comprises a plurality of storage nodes each comprising one or more storage devices. At least a given one of the storage nodes is configured to generate data blocks for prefilling of at least a subset of the storage devices, to maintain a first portion of the generated data blocks in a pool of data blocks available in the given storage node for prefilling one or more of the storage devices of the given storage node, and to send a second portion of the generated data blocks from the given storage node to one or more other ones of the storage nodes for prefilling one or more other ones of the storage devices of the one or more other storage nodes. Each of the other storage nodes can similarly maintain a separate pool of generated data blocks for prefilling its corresponding storage devices.
1. An apparatus comprising:
a storage system comprising a plurality of storage nodes each comprising one or more storage devices;
each of the storage nodes further comprising a processor coupled to a memory;
at least a given one of the storage nodes being configured:
to generate data blocks for prefilling of at least a subset of the storage devices;
to maintain a first portion of the generated data blocks in a pool of data blocks available in the given storage node for prefilling one or more of the storage devices of the given storage node; and
to send a second portion of the generated data blocks from the given storage node to one or more other ones of the storage nodes for prefilling one or more other ones of the storage devices of the one or more other storage nodes.
2. The apparatus of claim 1 wherein the storage system is configured to store the data blocks across the storage devices of the storage nodes utilizing a redundant array of independent disks (RAID) arrangement.
3. The apparatus of claim 2 wherein the RAID arrangement includes parity information supporting at least one recovery option for reconstructing the data blocks of at least one of the storage devices responsive to a failure of that storage device.
4. The apparatus of claim 1 wherein each of the storage nodes further comprises a set of processing modules configured to communicate over one or more networks with corresponding sets of processing modules on other ones of the storage nodes.
5. The apparatus of claim 4 wherein the sets of processing modules comprise respective servers that collectively implement at least a portion of a distributed storage controller of the storage system.
6. The apparatus of claim 4 wherein the sets of processing modules of the storage nodes each comprise at least one data module and at least one control module.
7. The apparatus of claim 1 wherein generating data blocks for prefilling of at least a subset of the storage devices comprises randomly generating the data blocks.
8. The apparatus of claim 1 wherein the given one of the storage nodes is further configured:
to monitor the pool of data blocks available in the given storage node for prefilling one or more of the storage devices of the given storage node; and
responsive to the number of data blocks in the pool being at or below a designated threshold, to send a request to one or more of the other storage nodes for additional data blocks generated by the one or more other storage nodes.
9. The apparatus of claim 8 wherein each of the other storage nodes generates one or more of the additional data blocks and sends those one or more additional data blocks to the given storage node.
10. The apparatus of claim 6 wherein different portions of a logical address space of the storage system are assigned to respective different ones of the control modules of the storage nodes, and wherein different portions of a content-based signature space of the storage system are assigned to respective different ones of the data modules of the storage nodes.
11. The apparatus of claim 10 wherein the content-based signature space of the storage system comprises a hash handle space of the storage system.
12. The apparatus of claim 10 wherein the content-based signature space of the storage system comprises a hash digest space of the storage system.
13. The apparatus of claim 1 wherein the given one of the storage nodes is further configured:
to compute a content-based signature for a particular one of the generated data blocks; and
responsive to the computed content-based signature for the particular one of the generated data blocks being within a portion of a content-based signature space assigned to the given storage node, to place the particular generated data block in the pool of data blocks available in the given storage node for prefilling one or more of the storage devices of the given storage node.
14. The apparatus of claim 1 wherein the given one of the storage nodes is further configured:
to compute a content-based signature for a particular one of the generated data blocks;
responsive to the content-based signature for the particular one of the generated data blocks not being within a portion of a content-based signature space assigned to the given storage node, to identify another one of the storage nodes for which the content-based signature for the particular one of the generated data blocks is within a portion of the content-based signature space assigned to that other storage node; and
to send the particular generated data block to that other storage node.
15. A method comprising:
configuring a storage system to include a plurality of storage nodes each comprising one or more storage devices, each of the storage nodes further comprising a processor coupled to a memory;
generating, in at least a given one of the storage nodes, a plurality of data blocks for prefilling of at least a subset of the storage devices;
maintaining a first portion of the generated data blocks in a pool of data blocks available in the given storage node for prefilling one or more of the storage devices of the given storage node; and
sending a second portion of the generated data blocks from the given storage node to one or more other ones of the storage nodes for prefilling one or more other ones of the storage devices of the one or more other storage nodes.
16. The method of claim 15 further comprising:
computing a content-based signature for a particular one of the generated data blocks; and
responsive to the computed content-based signature for the particular one of the generated data blocks being within a portion of a content-based signature space assigned to the given storage node, placing the particular generated data block in the pool of data blocks available in the given storage node for prefilling one or more of the storage devices of the given storage node.
17. The method of claim 15 further comprising:
computing a content-based signature for a particular one of the generated data blocks;
responsive to the content-based signature for the particular one of the generated data blocks not being within a portion of a content-based signature space assigned to the given storage node, identifying another one of the storage nodes for which the content-based signature for the particular one of the generated data blocks is within a portion of the content-based signature space assigned to that other storage node; and
sending the particular generated data block to that other storage node.
18. A computer program product comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code when executed by a storage system comprising a plurality of storage nodes each comprising one or more storage devices, each of the storage nodes further comprising a processor coupled to a memory, causes the storage system:
to generate, in at least a given one of the storage nodes, a plurality of data blocks for prefilling of at least a subset of the storage devices;
to maintain a first portion of the generated data blocks in a pool of data blocks available in the given storage node for prefilling one or more of the storage devices of the given storage node; and
to send a second portion of the generated data blocks from the given storage node to one or more other ones of the storage nodes for prefilling one or more other ones of the storage devices of the one or more other storage nodes.
19. The computer program product of claim 18 wherein the program code when executed by the storage system further causes the storage system:
to compute a content-based signature for a particular one of the generated data blocks; and
responsive to the computed content-based signature for the particular one of the generated data blocks being within a portion of a content-based signature space assigned to the given storage node, to place the particular generated data block in the pool of data blocks available in the given storage node for prefilling one or more of the storage devices of the given storage node.
20. The computer program product of claim 18 wherein the program code when executed by the storage system further causes the storage system:
to compute a content-based signature for a particular one of the generated data blocks;
responsive to the content-based signature for the particular one of the generated data blocks not being within a portion of a content-based signature space assigned to the given storage node, to identify another one of the storage nodes for which the content-based signature for the particular one of the generated data blocks is within a portion of the content-based signature space assigned to that other storage node; and
to send the particular generated data block to that other storage node.