Systems and methods for intelligently distributing data in a network scalable cluster using a cluster volume table (CVT) identifying owner storage nodes for logical blocks
Systems, methods, and computer-readable media for intelligent distribution of data in a storage cluster are described herein. An example method includes maintaining a cluster volume table (CVT) that stores information regarding data distribution of a volume across a plurality of storage nodes, and defining data distribution groups within the CVT. The CVT includes a plurality of entries, where each entry identifies an owner storage node for a respective logical block of the volume. Each data distribution group includes consecutive CVT entries. The method also includes receiving an input/output (I/O) operation directed to a region of the volume, identifying an owner storage node associated with the region of the volume using the CVT, and serving the I/O operation with at least one storage node of a data distribution group associated with the owner storage node. The I/O operation is served at a chunk-level granularity that is finer than a logical-block-level granularity of the volume. The at least one storage node that serves the I/O operation may be identified by performing a division and modulo operation. A modulus may equal the number of storage nodes in the data distribution group associated with the owner storage node. Data may be striped at the chunk-level granularity across the storage nodes in the data distribution group associated with the owner storage node.
1. A computer-implemented method for intelligent distribution of data in a storage cluster, comprising:
maintaining a cluster volume table (CVT) that stores information regarding data distribution of a volume across a plurality of storage nodes of the storage cluster, the CVT comprising a plurality of entries, wherein each of the entries identifies a respective owner storage node for a respective logical block of the volume;
defining a plurality of data distribution groups within the CVT, wherein each of the data distribution groups comprises a plurality of consecutive CVT entries;
receiving an input/output (I/O) operation directed to a region of the volume;
using the CVT, identifying an owner storage node associated with the region of the volume; and
serving the I/O operation with at least one storage node of a data distribution group associated with the identified owner storage node, wherein the I/O operation is served at a chunk-level granularity that is finer than a logical-block-level granularity of the volume.
2. The computer-implemented method of claim 1 , further comprising identifying the at least one storage node that serves the I/O operation by performing a division and modulo operation on a volume logical block address.
3. The computer-implemented method of claim 2 , wherein a modulus equals the number of storage nodes in the data distribution group associated with the identified owner storage node.
4. The computer-implemented method of claim 1 , wherein serving the I/O operation with at least one storage node of a data distribution group associated with the identified owner storage node further comprises striping data at the chunk-level granularity across the storage nodes in the data distribution group associated with the identified owner storage node.
5. The computer-implemented method of claim 1 , wherein each of the data distribution groups comprises entries identifying at least two different owner storage nodes.
6. The computer-implemented method of claim 1 , wherein a capacity of the volume is a multiple of a capacity of a data distribution group.
7. The computer-implemented method of claim 1 , wherein a capacity of a data distribution group is less than a minimum volume size.
8. The computer-implemented method of claim 1 , wherein each of the data distribution groups comprises a number of CVT entries greater than a number of consecutive blocks that can be allocated to a single storage node.
9. The computer-implemented method of claim 1 , wherein the plurality of storage nodes comprises at least two storage nodes having different capacities.
10. A non-transitory computer-readable recording medium having computer-executable instructions stored thereon for intelligent distribution of data in a storage cluster that, when executed by a processor of a storage system computer, cause the processor to:
maintain a cluster volume table (CVT) that stores information regarding data distribution of a volume across a plurality of storage nodes of the storage cluster, the CVT comprising a plurality of entries, wherein each of the entries identifies a respective owner storage node for a respective logical block of the volume;
define a plurality of data distribution groups within the CVT, wherein each of the data distribution groups comprises a plurality of consecutive CVT entries;
receive an input/output (I/O) operation directed to a region of the volume;
using the CVT, identify an owner storage node associated with the region of the volume; and
serve the I/O operation with at least one storage node of a data distribution group associated with the identified owner storage node, wherein the I/O operation is served at a chunk-level granularity that is finer than a logical-block-level granularity of the volume.
11. The non-transitory computer-readable recording medium of claim 10 , having further computer-executable instructions stored thereon that, when executed by the processor, cause the processor to identify the at least one storage node that serves the I/O operation by performing a division and modulo operation on a volume logical block address.
12. The non-transitory computer-readable recording medium of claim 11 , wherein a modulus equals the number of storage nodes in the data distribution group associated with the identified owner storage node.
13. The non-transitory computer-readable recording medium of claim 10 , wherein serving the I/O operation with at least one storage node of a data distribution group associated with the identified owner storage node further comprises striping data at the chunk-level granularity across the storage nodes in the data distribution group associated with the identified owner storage node.
14. The non-transitory computer-readable recording medium of claim 10 , wherein each of the data distribution groups comprises entries identifying at least two different owner storage nodes.
15. The non-transitory computer-readable recording medium of claim 10 , wherein a capacity of the volume is a multiple of a capacity of a data distribution group.
16. The non-transitory computer-readable recording medium of claim 10 , wherein each of the data distribution groups comprises a number of CVT entries greater than a number of consecutive blocks that can be allocated to a single storage node.
17. The non-transitory computer-readable recording medium of claim 10 , wherein the plurality of storage nodes comprises at least two storage nodes having different capacities.
18. A storage node for intelligent distribution of data in a storage cluster, the storage node comprising:
a processor; and
a memory operably coupled to the processor, wherein the memory has computer-executable instructions stored thereon that, when executed by the processor, cause the processor to:
maintain a cluster volume table (CVT) that stores information regarding data distribution of a volume across a plurality of storage nodes of the storage cluster, the CVT comprising a plurality of entries, wherein each of the entries identifies a respective owner storage node for a respective logical block of the volume;
define a plurality of data distribution groups within the CVT, wherein each of the data distribution groups comprises a plurality of consecutive CVT entries;
receive an input/output (I/O) operation directed to a region of the volume;
using the CVT, identify an owner storage node associated with the region of the volume; and
serve the I/O operation with at least one storage node of a data distribution group associated with the identified owner storage node, wherein the I/O operation is served at a chunk-level granularity that is finer than a logical-block-level granularity of the volume.
19. The storage node of claim 18 , wherein the memory has further computer-executable instructions stored thereon that, when executed by the processor, cause the processor to identify the at least one storage node that serves the I/O operation by performing a division and modulo operation on a volume logical block address.
20. The storage node of claim 19 , wherein a modulus equals the number of storage nodes in the data distribution group associated with the identified owner storage node.