Aggregated background processing in a data storage system to improve system resource utilization
View Patent ↗Techniques for aggregating background processing in a data storage system. Blocks are identified having contents on which a data operation was not performed in-line. The background data operation is prevented for blocks that will no longer be accessed by the host computer because they are only mapped to files implementing data objects that are scheduled for future deletion. A region of blocks may be selected that meets a criteria for performing a background free space operation, and the background data operation may be performed on the contents of blocks in the selected region while the contents of those blocks are being relocated to other blocks while performing the background free space operation. While performing the background data operation, blocks may be freed from files that implement data objects scheduled for future deletion.
1. A method of aggregating background processing performed in a data storage system, comprising the steps of:
providing blocks within an address space of a file system, wherein the file system is internal to the data storage system, wherein the file system maps files to the blocks in the address space, wherein each file implements a corresponding data object that is accessible to at least one host computer, and wherein blocks in the address space also correspond to non-volatile storage provisioned from a set of storage devices that provide storage for the file system;
identifying blocks within the address space having contents on which a background data operation has not previously been performed; and
while performing the background data operation on the contents of the blocks identified within the address space as having contents on which the background data operation has not previously been performed, i) identifying blocks having contents on which the background data operation has not previously been performed and that will no longer be accessed by the host computer, and ii) preventing the background data operation from being performed on blocks having contents on which the data operation has not previously been performed and that will no longer be accessed by the host computer.
2. The method of claim 1 , further comprising:
selecting a region of blocks within the address space that meets criteria for performing a background free space operation;
wherein identifying blocks within the address space having contents on which the background data operation has not previously been performed is performed within the selected region; and
wherein performing the background data operation on the contents of blocks identified within the selected region as having contents on which the background data operation has not previously been performed is performed while the contents of the blocks identified within the selected region as having contents on which the background data operation has not previously been performed are being relocated as part of performing the background free space operation.
3. The method of claim 2 , wherein meeting the criteria for performing the background free space operation comprises having less than a threshold percentage of allocated blocks; and
wherein selecting the region of blocks within the address space that meets the criteria for performing the background free space operation comprises selecting a region of blocks having less than a threshold percentage of allocated blocks and corresponding to a slice of non-volatile storage that is provisioned from the set of storage devices and that is used as non-volatile storage for the contents of the blocks in the region.
4. The method of claim 1 , wherein identifying blocks having contents on which the background data operation has not previously been performed and that will no longer be accessed by the host computer includes:
for each block having contents on which the background data operation has not previously been performed, i) finding a set of one or more files that are mapped to the block, and ii) identifying the block as having contents on which the background data operation has not previously been performed and that will no longer be accessed by the host in response to determining that all files mapped to the block implement data objects that are scheduled for future deletion.
5. The method of claim 1 , wherein performing the data operation on the contents of blocks identified within the address space as having contents on which the background data operation has not previously been performed includes, further performing the following steps for each identified block:
finding a set of one or more files that are mapped to the block;
determining that at least one of the files mapped to the block implements a data object that is scheduled for future deletion; and
freeing the block from the file mapped to the block that implements the data object that is scheduled for future deletion, such that the file that implements the data object that is scheduled for further deletion is no longer mapped to the block.
6. The method of claim 1 , wherein the background data operation comprises data deduplication; and
wherein performing the background data operation on the contents of blocks identified within the address space as having contents on which the background data operation has not previously been performed includes performing data deduplication on the contents of blocks identified within the address space as having contents on which data deduplication has not previously been performed.
7. The method of claim 1 , wherein the background data operation comprises data compression; and
wherein performing the background data operation on the contents of blocks identified within the address space as having contents on which the background data operation has not previously been performed includes performing data compression on the contents of blocks identified within the address space as having contents on which data compression has not previously been performed.
8. The method of claim 1 , wherein the background data operation comprises data encryption; and
wherein performing the background data operation on the contents of blocks identified within the address space as having contents on which the background data operation has not previously been performed includes performing data encryption on the contents of blocks identified within the address space as having contents on which data encryption has not previously been performed.
9. The method of claim 2 , wherein the background free space operation comprises free space defragmentation; and
wherein performing the data operation on the contents of blocks identified within the selected region as having contents on which the background data operation has not previously been performed is performed while the contents of the blocks identified within the selected region as having contents on which the background data operation has not previously been performed are being relocated as part of performing free space defragmentation.
10. A computerized apparatus, comprising a set of processing units and memory, coupled to the set of processing units, the memory storing instructions for aggregating background processing which, when executed by the set of processing units, cause the set of processing units to:
provide blocks within an address space of a file system, wherein the file system is internal to the data storage system, wherein the file system maps files to the blocks in the address space, wherein each file implements a corresponding data object that is accessible to at least one host computer, and wherein blocks in the address space also correspond to non-volatile storage provisioned from a set of storage devices that provide storage for the file system;
identify blocks within the address space having contents on which a background data operation has not previously been performed; and
while performing the background data operation on the contents of the blocks identified within the address space as having contents on which the background data operation has not previously been performed, i) identify blocks having contents on which the background data operation has not previously been performed and that will no longer be accessed by the host computer, and ii) prevent the background data operation from being performed on blocks having contents on which the data operation has not previously been performed and that will no longer be accessed by the host computer.
11. The computerized apparatus of claim 10 , wherein the instructions for aggregating background processing, when executed by the set of processing units, further cause the set of processing units to:
select a region of blocks within the address space that meets criteria for performing a background free space operation;
identify blocks within the address space having contents on which the background data operation has not previously been performed by identifying blocks on which the background data operation has not previously been performed within the selected region; and
wherein the background data operation is performed on the contents of blocks identified within the selected region as having contents on which the background data operation has not previously been performed while the contents of the blocks identified within the selected region as having contents on which the background data operation has not previously been performed are being relocated as part of performing the background free space operation.
12. The computerized apparatus of claim 11 , wherein meeting the criteria for performing the background free space operation comprises having less than a threshold percentage of allocated blocks; and
wherein the region of blocks is selected within the address space that meets the criteria for performing the background free space operation by selecting a region of blocks having less than a threshold percentage of allocated blocks and corresponding to a slice of non-volatile storage that is provisioned from the set of storage devices and that is used as non-volatile storage for the contents of the blocks in the region.
13. The computerized apparatus of claim 10 , wherein the blocks identified as having contents on which the background data operation has not previously been performed and that will no longer be accessed by the host computer are identified by:
for each block having contents on which the background data operation has not previously been performed, i) finding a set of one or more files that are mapped to the block, and ii) identifying the block as having contents on which the background data operation has not previously been performed and that will no longer be accessed by the host in response to determining that all files mapped to the block implement data objects that are scheduled for future deletion.
14. The computerized apparatus of claim 10 , wherein the data operation is performed on the contents of blocks identified within the address space as having contents on which the background data operation has not previously been performed at least in part by, for each identified block:
finding a set of one or more files that are mapped to the block;
determining that at least one of the files mapped to the block implements a data object that is scheduled for future deletion; and
freeing the block from the file mapped to the block that implements the data object that is scheduled for future deletion, such that the file that implements the data object that is scheduled for further deletion is no longer mapped to the block.
15. The computerized apparatus of claim 10 , wherein the background data operation comprises data deduplication; and
wherein the background data operation is performed on the contents of blocks identified within the address space as having contents on which the background data operation has not previously been performed at least in part by performing data deduplication on the contents of blocks identified within the address space as having contents on which data deduplication has not previously been performed.
16. The computerized apparatus of claim 10 , wherein the background data operation comprises data compression; and
wherein the background data operation is performed on the contents of blocks identified within the address space as having contents on which the background data operation has not previously been performed at least in part by performing data compression on the contents of blocks identified within the address space as having contents on which data compression has not previously been performed.
17. The computerized apparatus of claim 10 , wherein the background data operation comprises data encryption; and
wherein the background data operation is performed on the contents of blocks identified within the address space as having contents on which the background data operation has not previously been performed at least in part by performing data encryption on the contents of blocks identified within the address space as having contents on which data encryption has not previously been performed.
18. The method of claim 11 , wherein the background free space operation comprises free space defragmentation; and
wherein the data operation performed on the contents of blocks identified within the selected region as having contents on which the background data operation has not previously been performed is performed while the contents of the blocks identified within the selected region as having contents on which the background data operation has not previously been performed are being relocated as part of performing free space defragmentation.
19. A non-transitory, computer-readable medium including instructions which, when executed by a set of processing units of a computerized apparatus, cause the set of processing units to perform a method for aggregating background processing performed in a data storage system, the method comprising the steps of:
providing blocks within an address space of a file system, wherein the file system is internal to the data storage system, wherein the file system maps files to the blocks in the address space, wherein each file implements a corresponding data object that is accessible to at least one host computer, and wherein blocks in the address space also correspond to non-volatile storage provisioned from a set of storage devices that provide storage for the file system;
identifying blocks within the address space having contents on which a background data operation has not previously been performed; and
while performing the background data operation on the contents of the blocks identified within the address space as having contents on which the background data operation has not previously been performed, i) identifying blocks having contents on which the background data operation has not previously been performed and that will no longer be accessed by the host computer, and ii) preventing the background data operation from being performed on blocks having contents on which the data operation has not previously been performed and that will no longer be accessed by the host computer.