Data storage system employing space reclaim approximation
View Patent ↗A method of conditionally truncating a volume file in a data storage system includes identifying counts of categories of data blocks of the volume file, including (1) a first count of blocks owned by the volume file and located after the target end-of-file, (2) a second count of blocks located after the target end-of-file and requiring relocation to before the target end-of-file, and (3) a third count of blocks being free and owned by the volume file and located before the target end-of-file. An approximation of space to be freed is generated as a difference between the first count and a value expressing an excess of the second count over the third count. When the approximation is above a predetermined threshold, then the volume file is truncated to the target end-of-file, and otherwise truncating is refrained from.
1. A method of operating a data storage system having physical storage devices and one or more storage processors executing computer program instructions, comprising:
defining an upper file system, a lower file system, and an intermediate logical volume, the intermediate logical volume viewed as an underlying storage volume by the upper file system and stored as a set of volume files of the lower file system, the volume files including a primary volume file for a current version of the storage volume and one or more secondary volume files for snapshot copies of the storage volume at past points in time, the method directed to conditionally truncating the primary volume file to a target end-of-file based upon an approximation of an amount of storage space to be returned to an underlying storage pool;
identifying respective counts of categories of data blocks of the primary volume file, including (1) a first count of blocks being owned by the primary volume file and located after the target end-of-file, (2) a second count of blocks being located after the target end-of-file and requiring relocation to before the target end-of-file, and (3) a third count of blocks being free and owned by the primary volume file and located before the target end-of-file;
generating an approximation of space to be freed as a difference between the first count and a value expressing an excess of the second count over the third count; and
when the approximation is above a predetermined threshold, then truncating the primary volume file to the target end-of-file, and otherwise refraining from truncating the primary volume file to the target end-of-file.
2. The method of claim 1 , wherein the first count of blocks includes a count of free owned blocks and a count of in-use owned blocks, and the second count of blocks includes the count of in-use owned blocks and a count of in-use shared blocks shared with one or more of the snapshot copies of the storage volume.
3. The method of claim 1 , wherein the second count of blocks includes a count of in-use owned blocks and a count of in-use shared blocks shared with one or more of the snapshot copes of the storage volume, and the first count of blocks includes a count of free owned blocks and the count of in-use owned blocks.
4. The method of claim 1 , wherein the value expressing an excess of the second count over the third count is calculated by a conditional function returning a difference between the second count and the third count if the difference is non-negative, and returning zero otherwise.
5. The method of claim 1 , wherein the identifying the respective counts includes reverse-mapping blocks of the primary volume file to the upper file system to identify whether the blocks are free or in-use.
6. The method of claim 1 , wherein the identifying the respective counts includes (a) resolving each block to a block number in the primary volume file and reading block metadata of the block, (b) comparing a weight value from the block metadata to a weight value from a mapping pointer for the block, and (c) identifying the block as owned when the weight value of the mapping pointer is equal to the weight value in the block metadata, and otherwise identifying the block as shared.
7. A data storage system, comprising:
physical storage devices;
respective interfaces to the physical storage devices and to a network via which client storage requests are received; and
one or more storage processors coupled to the physical storage devices and to the network via the respective interfaces, the storage processors configured and operative to perform a method including:
defining an upper file system, a lower file system, and an intermediate logical volume, the intermediate logical volume viewed as an underlying storage volume by the upper file system and stored as a set of volume files of the lower file system, the volume files including a primary volume file for a current version of the storage volume and one or more secondary volume files for snapshot copies of the storage volume at past points in time, the method directed to conditionally truncating the primary volume file to a target end-of-file based upon an approximation of an amount of storage space to be returned to an underlying storage pool;
identifying respective counts of categories of data blocks of the primary volume file, including (1) a first count of blocks being owned by the primary volume file and located after the target end-of-file, (2) a second count of blocks being located after the target end-of-file and requiring relocation to before the target end-of-file, and (3) a third count of blocks being free and owned by the primary volume file and located before the target end-of-file;
generating an approximation of space to be freed as a difference between the first count and a value expressing an excess of the second count over the third count; and
when the approximation is above a predetermined threshold, then truncating the primary volume file to the target end-of-file, and otherwise refraining from truncating the primary volume file to the target end-of-file.
8. The data storage system of claim 7 , wherein the first count of blocks includes a count of free owned blocks and a count of in-use owned blocks, and the second count of blocks includes the count of in-use owned blocks and a count of in-use shared blocks shared with one or more of the snapshot copies of the storage volume.
9. The data storage system of claim 7 , wherein the second count of blocks includes a count of in-use owned blocks and a count of in-use shared blocks shared with one or more of the snapshot copes of the storage volume, and the first count of blocks includes a count of free owned blocks and the count of in-use owned blocks.
10. The data storage system of claim 7 , wherein the value expressing an excess of the second count over the third count is calculated by a conditional function returning a difference between the second count and the third count if the difference is non-negative, and returning zero otherwise.
11. The data storage system claim 7 , wherein the identifying the respective counts includes reverse-mapping blocks of the primary volume file to the upper file system to identify whether the blocks are free or in-use.
12. The data storage system of claim 7 , wherein the identifying the respective counts includes (a) resolving each block to a block number in the primary volume file and reading block metadata of the block, (b) comparing a weight value from the block metadata to a weight value from mapping pointer for the block, and (c) identifying the block as owned when the weight value in the block metadata, and otherwise identifying the block as shared.
13. A non-transitory computer-readable medium storing computer program instructions, the instructions being executable by a set of one or more storage processors of a data storage system to cause the data storage system to perform a method including:
defining an upper file system, a lower file system, and an intermediate logical volume, the intermediate logical volume viewed as an underlying storage volume by the upper file system and stored as a set of volume files of the lower file system, the volume files including a primary volume file for a current version of the storage volume and one or more secondary volume files for snapshot copies of the storage volume at past points in time, the method directed to conditionally truncating the primary volume file to a target end-of-file based upon an approximation of an amount of storage space to be returned to an underlying storage pool;
identifying respective counts of categories of data blocks of the primary volume file, including (1) a first count of blocks being owned by the primary volume file and located after the target end-of-file, (2) a second count of blocks being located after the target end-of-file and requiring relocation to before the target end-of-file, and (3) a third count of blocks being free and owned by the primary volume file and located before the target end-of-file;
generating an approximation of space to be freed as a difference between the first count and a value expressing an excess of the second count over the third count; and
when the approximation is above a predetermined threshold, then truncating the primary volume file to the target end-of-file, and otherwise refraining from truncating the primary volume file to the target end-of-file.
14. The non-transitory computer-readable medium of claim 13 , wherein the first count of blocks includes a count of free owned blocks and a count of in-use owned blocks, and the second count of blocks includes the count of in-use owned blocks and a count of in-use shared blocks shared with one or more of the snapshot copies of the storage volume.
15. The non-transitory computer-readable medium of claim 13 , wherein the second count of blocks includes a count of in-use owned blocks and a count of in-use shared blocks shared with one or more of the snapshot copes of the storage volume, and the first count of blocks includes a count of free owned blocks and the count of in-use owned blocks.
16. The non-transitory computer-readable medium of claim 13 , wherein the value expressing an excess of the second count over the third count is calculated by a conditional function returning a difference between the second count and the third count if the difference is non-negative, and returning zero otherwise.
17. The non-transitory computer-readable medium of claim 13 , wherein the identifying the respective counts includes reverse-mapping blocks of the primary volume file to the upper file system to identify whether the blocks are free or in-use.
18. The non-transitory computer-readable medium of claim 13 , wherein the identifying the respective counts includes (a) resolving each block to a block number in the primary volume file and reading block metadata of the block, (b) comparing a weight value from the block metadata to a weight value from a mapping pointer for the block, and (c) identifying the block as owned when the weight value of the mapping pointer is equal to the weight value in the block metadata, and otherwise identifying the block as shared.