Techniques for rebalancing storage between subspaces
Techniques for improving storage efficiency of a data storage system are provided. The techniques include (a) performing a testing operation to evaluate whether a test condition is satisfied for a filesystem having a plurality of subspaces, each subspace being provisioned with a respective amount of storage, each subspace of the filesystem being dedicated to storing different respective types of filesystem content, wherein the test condition indicates that a first subspace has available space in excess of a threshold percentage, (b) in response to determining that the test condition is satisfied for the filesystem, performing a reduction operation on the first subspace to remove a portion of storage provisioned to that subspace while retaining the removed portion for use by the filesystem, and (c) upon a second subspace reaching a fullness threshold, provisioning the removed portion to the second subspace and storing content of the respective type of the second subspace thereon.
1. A method of improving storage efficiency of a data storage system, the method comprising:
performing a testing operation to evaluate whether a test condition is satisfied for a filesystem hosted by the data storage system, the filesystem having a plurality of subspaces, each subspace of the filesystem being provisioned with a respective amount of storage, each subspace of the filesystem being dedicated to storing different respective types of filesystem content, wherein the test condition indicates that a first subspace of the plurality of subspaces has available space in excess of a threshold percentage;
in response to determining that the test condition is satisfied for the filesystem, performing a reduction operation on the first subspace to remove a portion of storage provisioned to that subspace while retaining the removed portion of storage for use by the filesystem; and
upon a second subspace of the plurality of subspaces reaching a fullness threshold, provisioning the removed portion of storage to the second subspace and storing content of the respective type of the second subspace thereon; wherein:
the test condition further indicates that a third subspace of the plurality of subspaces has available space below a minimum threshold percentage, the third subspace being different than the first subspace; and
the second subspace reaching the fullness threshold indicates that the second subspace has less than a critical amount of available storage, the critical amount of storage being less than the minimum threshold percentage.
2. The method of claim 1 wherein performing the testing operation to evaluate whether the test condition is satisfied is done periodically.
3. The method of claim 2 ,
wherein the method further comprises periodically performing a filesystem shrink operation at a first frequency, the filesystem shrink operation being configured to rearrange content of the filesystem into a more compact arrangement and to return unused storage provisioned to the filesystem to no longer be provisioned to the filesystem; and
wherein periodically performing the testing operation is performed at a second frequency, the second frequency being more frequent than the first frequency.
4. The method of claim 1 wherein performing the testing operation to evaluate whether the test condition is satisfied is done in response to a deletion of content from the filesystem.
5. The method of claim 1 ,
wherein the filesystem is one of a plurality of filesystems hosted on the data storage system; and
performing the testing operation is performed for each filesystem of the plurality of filesystems by a respective thread assigned to that filesystem.
6. The method of claim 1 wherein the filesystem is configured with an intermediate threshold percentage and a differential value, the threshold percentage being equal to the intermediate threshold percentage plus the differential value and the minimum threshold percentage being equal to the intermediate threshold percentage minus the differential value, wherein a given subspace having an available space below the intermediate threshold percentage triggers an attempt to provision the filesystem with more storage.
7. The method of claim 1 wherein the filesystem is a thickly-provisioned filesystem.
8. The method of claim 1 ,
wherein the respective amount of storage provisioned to each subspace is a respective integer multiple of a size of an allocation slice, an allocation slice being a smallest unit of storage that can be provisioned to the filesystem, each allocation slice including a plurality of contiguously-arranged blocks of content;
wherein removing the portion of storage provisioned to the first subspace includes:
moving all content from a first allocation slice of the first subspace to one or more other allocation slices of the first subspace; and
removing the first allocation slice from being provisioned to the first subspace and storing the first allocation slice within a pool of available allocation slices for the filesystem; and
wherein provisioning the removed portion of storage to the second subspace includes:
removing the first allocation slice from the pool of available allocation slices for the filesystem; and
provisioning the first allocation slice to the second subspace.
9. The method of claim 8 wherein each allocation slice has a fixed size in the range of 128 megabytes to 256 gigabytes.
10. The method of claim 1 wherein the plurality of subspaces of the filesystem includes:
a data subspace dedicated to storing file data; and
a metadata subspace assigned to store indirect blocks that include pointers to blocks of file data.
11. The method of claim 10 , wherein the plurality of subspaces of the filesystem further includes an inode subspace dedicated to storing inodes that define files of the filesystem.
12. The method of claim 10 , wherein the plurality of subspaces of the filesystem further includes a shadow subspace assigned to store descriptive information about layout and allocation of data blocks within the data subspace.
13. A computer program product comprising a non-transitory computer-readable storage medium storing a set of instructions, which, when executed by control circuitry of a data storage system, cause the control circuitry to improve storage efficiency of the data storage system by:
performing a testing operation to evaluate whether a test condition is satisfied for a filesystem hosted by the data storage system, the filesystem having a plurality of subspaces, each subspace of the filesystem being provisioned with a respective amount of storage, each subspace of the filesystem being dedicated to storing different respective types of filesystem content, wherein the test condition indicates that a first subspace of the plurality of subspaces has available space in excess of a threshold percentage;
in response to determining that the test condition is satisfied for the filesystem, performing a reduction operation on the first subspace to remove a portion of storage provisioned to that subspace while retaining the removed portion of storage for use by the filesystem; and
upon a second subspace of the plurality of subspaces reaching a fullness threshold, provisioning the removed portion of storage to the second subspace and storing content of the respective type of the second subspace thereon;
wherein the respective amount of storage provisioned to each subspace is a respective integer multiple of a size of an allocation slice, an allocation slice being a smallest unit of storage that can be provisioned to the filesystem, each allocation slice including a plurality of contiguously-arranged blocks of content;
wherein removing the portion of storage provisioned to the first subspace includes:
moving all content from a first allocation slice of the first subspace to one or more other allocation slices of the first subspace; and
removing the first allocation slice from being provisioned to the first subspace and storing the first allocation slice within a pool of available allocation slices for the filesystem; and
wherein provisioning the removed portion of storage to the second subspace includes:
removing the first allocation slice from the pool of available allocation slices for the filesystem; and
provisioning the first allocation slice to the second subspace.
14. The computer program product of claim 13 wherein the test condition further indicates that a third subspace of the plurality of subspaces has available space below a minimum threshold percentage, the third subspace being different than the first subspace.
15. The computer program product of claim 14 wherein the filesystem is configured with an intermediate threshold percentage and a differential value, the threshold percentage being equal to the intermediate threshold percentage plus the differential value and the minimum threshold percentage being equal to the intermediate threshold percentage minus the differential value, wherein a given subspace having an available space below the intermediate threshold percentage triggers an attempt to provision the filesystem with more storage.
16. An apparatus comprising:
persistent storage devices providing data storage;
network interface circuitry configured to communicate with a host over a network; and
processing circuitry coupled to memory to form a control circuit constructed and arranged to improve storage efficiency by:
performing a testing operation to evaluate whether a test condition is satisfied for a filesystem hosted by the apparatus, the filesystem having a plurality of subspaces, each subspace of the filesystem being provisioned with a respective amount of storage, each subspace of the filesystem being dedicated to storing different respective types of filesystem content, wherein the test condition indicates that a first subspace of the plurality of subspaces has available space in excess of a threshold percentage;
in response to determining that the test condition is satisfied for the filesystem, performing a reduction operation on the first subspace to remove a portion of storage provisioned to that subspace while retaining the removed portion of storage for use by the filesystem; and
upon a second subspace of the plurality of subspaces reaching a fullness threshold, provisioning the removed portion of storage to the second subspace and storing content of the respective type of the second subspace thereon;
wherein:
the test condition further indicates that a third subspace of the plurality of subspaces has available space below a minimum threshold percentage, the third subspace being different than the first subspace; and
the filesystem is configured with an intermediate threshold percentage and a differential value, the threshold percentage being equal to the intermediate threshold percentage plus the differential value and the minimum threshold percentage being equal to the intermediate threshold percentage minus the differential value, wherein a given subspace having an available space below the intermediate threshold percentage triggers an attempt to provision the filesystem with more storage.
17. The method of claim 1 wherein the test condition indicates that the third subspace has available space below the minimum threshold percentage prior to the second subspace reaching the fullness threshold.
18. The computer program product of claim 13 wherein each allocation slice has a fixed size in the range of 128 megabytes to 256 gigabytes.