Systems and methods for implementing space consolidation and space expansion in a horizontally federated cluster

1. A computer-implemented method for consolidating space in a storage cluster, comprising:

striping a volume across a plurality of storage nodes, wherein a plurality of logical blocks of the volume are distributed to the storage nodes;

maintaining a cluster volume table (CVT) storing information regarding distribution of the logical blocks across the storage nodes, wherein the CVT comprises a plurality of entries, and wherein each of the entries in the CVT comprises information identifying a respective owner storage node of a respective logical block;

adding a new storage node to the storage cluster, wherein a portion of the new storage node's storage capacity is reserved for space consolidation;

transferring data for one or more logical blocks from each of the storage nodes to the new storage node based on respective usage ratios of the storage nodes;

updating the CVT to identify the new storage node as owner storage node of the one or more logical blocks transferred to the new storage node;

receiving a list of logical blocks eligible for transfer from the storage nodes to the new storage node, wherein a logical block eligible for transfer is not associated with a plurality of snapshot versions; and

populating a transfer array comprising the list of logical blocks eligible for transfer.

2. The computer-implemented method of claim 1 , further comprising scanning the storage nodes to determine the respective usage ratios of the storage nodes.

3. The computer-implemented method of claim 2 , further comprising normalizing the respective usage ratios of the storage nodes.

4. The computer-implemented method of claim 3 , wherein the data for one or more logical blocks is transferred to the new storage node based on the normalized respective usage ratios of the storage nodes.

5. The computer-implemented method of claim 1 , wherein the transfer array comprises a plurality of non-contiguous logical blocks.

6. The computer-implemented method of claim 1 , further comprising locking a particular logical block to be transferred to the new storage node, wherein the data for one or more logical blocks transferred to the new storage node comprises data for the particular logical block.

7. The computer-implemented method of claim 6 , further comprising:

notifying the storage nodes that the new storage node is the owner storage node for the particular logical block; and

unlocking the particular logical block.

8. The computer-implemented method of claim 1 , further comprising redistributing a plurality of unallocated logical blocks based on the respective usage ratios of the storage nodes.

9. The computer-implemented method of claim 3 , further comprising redistributing a plurality of unallocated logical blocks based on the normalized respective usage ratios of the storage nodes.

10. The computer-implemented method of claim 1 , wherein a first portion of the new storage node's storage capacity is reserved for space consolidation and a second portion of the new storage node's storage capacity is reserved for space redistribution.

11. The computer-implemented method of claim 1 , wherein each of the entries in the CVT further comprises information identifying a respective owner storage node for remotely-owned logical block or a pointer to a system volume table (SVT) for a locally-owned logical block.

12. A non-transitory computer-readable recording medium having computer-executable instructions stored thereon for consolidating space in a storage cluster that, when executed by a storage system computer, cause the storage system computer to:

stripe a volume across a plurality of storage nodes, wherein a plurality of logical blocks of the volume are distributed to the storage nodes;

maintain a cluster volume table (CVT) storing information regarding distribution of the logical blocks across the storage nodes, wherein the CVT comprises a plurality of entries, and wherein each of the entries in the CVT comprises information identifying a respective owner storage node of a respective logical block;

add a new storage node to the storage cluster, wherein a portion of the new storage node's storage capacity is reserved for space consolidation;

transfer data for one or more logical blocks from each of the storage nodes to the new storage node based on respective usage ratios of the storage nodes;

update the CVT to identify the new storage node as owner storage node of the one or more blocks transferred to the new storage node;

receive a list of logical blocks eligible for transfer from the storage nodes to the new storage node, wherein a logical block eligible for transfer is not associated with a plurality of snapshot versions; and

populating a transfer array comprising the list of logical blocks eligible for transfer.

13. The non-transitory computer-readable recording medium of claim 12 , having further computer-executable instructions stored thereon that, when executed by the storage system computer, cause the storage system computer to lock a particular logical block to be transferred to the new storage node, wherein the data for one or more logical blocks transferred to the new storage node comprises data for the particular logical block.

14. The non-transitory computer-readable recording medium of claim 12 , having further computer-executable instructions stored thereon that, when executed by the storage system computer, cause the storage system computer to redistribute a plurality of unallocated logical blocks.