Dynamic data compression
Techniques for performing compression operations on persistently-stored data blocks during read/write commands. A method embodiment performs in-line data compression operations over data blocks referenced by a caller. The in-line data compression operations are performed during execution of a storage input-output (I/O) command, between the event of receipt of the storage I/O command and the event of returning status of the storage I/O command. The storage I/O operation is associated with at least one data group comprising one or more data blocks that are identified by the caller. Upon receipt of the storage I/O command, one or more compression rules are applied to the data blocks to determine one or more compression parameters, which compression parameters are used to form specific compression operations that are performed over at least a portion of the data group. The status pertaining to the execution of the storage I/O operation is returned to the caller.
1. A method for performing one or more in-line data compression operations over one or more data units during execution of a storage input-output (IO) operation, the method comprising:
receiving at least one storage IO operation associated with at least one data group comprising one or more data units, the at least one storage IO operation being received from an application and comprising a read request or a write request;
applying one or more compression rules in-line with the at least one storage IO operation to the data units to determine one or more compression parameters, wherein actions performed in-line are actions completed prior to returning or writing the one or more data units at one or more storage locations specified by the at least one storage IO operation;
executing the at least one storage IO operation in accordance with the compression parameters, wherein the storage IO operation performs at least one in-line data compression operations on at least a portion of the data group, the at least one in-line data compression operations comprising compressing or decompressing at least the portion of the data group according to the compression parameters prior to returning or writing the one or more data units at one or more storage locations specified by the at least one storage IO operation; and
returning a status to the application, the status pertaining to the execution of the storage IO operation.
2. The method of claim 1 , wherein the at least one in-line data compression operations complete before returning the status pertaining to the execution of the storage IO operation to the application.
3. The method of claim 1 , further comprising performing a compression efficiency test over at least one of the data units to determine one or more data unit attributes associated with the at least one of the data units.
4. The method of claim 3 , wherein the data unit attributes comprise at least one of, a compression ratio, or a compression size.
5. The method of claim 3 , further comprising updating a set of metadata to describe the data unit attributes.
6. The method of claim 1 , further comprising:
collecting one or more data unit attributes associated with the data units; and
applying the compression rules in-line with the at least one storage IO operation to at least one of the data unit attributes to determine at least one of the compression parameters,
wherein the in-line data compression operations are based at least in part on the compression parameters.
7. The method of claim 1 , further comprising:
reading, from a storage facility, a stored representation; and
analyzing the stored representation to determine at least one of, one or more compression parameters, or one or more compression operations.
8. The method of claim 1 , further comprising partitioning the data group into the data units.
9. The method of claim 1 , wherein the storage IO operation comprises at least one of, write instruction, or a read instruction.
10. The method of claim 1 , wherein the compression parameters characterize at least one of, a storage device, a compression level, a compression type, a compression algorithm, a storage allocation cushion, or a compression delay.
11. The method of claim 1 , wherein the data units comprise at least one of, one or more extents, one or more slices, one or more blocks, one or more areas, or one or more regions.
12. A computer program, embodied in a non-transitory computer readable medium, the non-transitory computer readable medium having stored thereon a sequence of instructions which, when stored in memory and executed by one or more processors causes the one or more processor to execute a set of acts for performing one or more in-line data compression operations over one or more data units during execution of a storage input-output (IO) operation, the set of acts comprising:
receiving at least one storage IO operation associated with at least one data group comprising one or more data units, the at least one storage IO operation being received from an application and comprising a read request or a write request;
applying one or more compression rules in-line with the at least one storage IO operation to the data units to determine one or more compression parameters, wherein actions performed in-line are actions completed prior to returning or writing the one or more data units at one or more storage locations specified by the at least one storage IO operation;
executing the at least one storage IO operation in accordance with the compression parameters, wherein the storage IO operation performs at least one in-line data compression operations on at least a portion of the data group, the at least one in-line data compression operations comprising compressing or decompressing at least the portion of the data group according to the compression parameters prior to returning or writing the one or more data units at one or more storage locations specified by the at least one storage IO operation; and
returning a status to the application, the status pertaining to the execution of the storage IO operation.
13. The computer readable medium of claim 12 , wherein the at least one in-line data compression operations complete before returning the status pertaining to the execution of the storage IO operation to the application.
14. The computer readable medium of claim 12 , set of acts further comprising performing a compression efficiency test over at least one of the data units to determine one or more data unit attributes associated with the at least one of the data units.
15. The computer readable medium of claim 14 , wherein the data unit attributes comprise at least one of, a compression ratio, or a compression size.
16. The computer readable medium of claim 14 , the set of acts further comprising updating a set of metadata to describe the data unit attributes.
17. The computer readable medium of claim 12 , the set of acts further comprising:
collecting one or more data unit attributes associated with the data units; and
applying the compression rules in-line with the at least one storage IO operation to at least one of the data unit attributes to determine at least one of the compression parameters,
wherein the in-line data compression operations are based at least in part on the compression parameters.
18. The computer readable medium of claim 12 , the set of acts further comprising:
reading, from a storage facility, a stored representation; and
analyzing the stored representation to determine at least one of, one or more compression parameters, or one or more compression operations.
19. A system for performing one or more in-line data compression operations over one or more data units during execution of a storage input-output (IO) operation, the system comprising:
a storage medium having stored thereon a sequence of instructions; and
one or more processors that execute the sequence of instructions to cause the one or more processors to perform a set of acts, the set of acts comprising:
receiving at least one storage IO operation associated with at least one data group comprising one or more data units, the at least one storage IO operation being received from an application and comprising a read request or a write request;
applying one or more compression rules in-line with the at least one storage IO operation to the data units to determine one or more compression parameters, wherein actions performed in-line are actions completed prior to returning or writing the one or more data units at one or more storage locations specified by the at least one storage IO operation;
executing the at least one storage IO operation in accordance with the compression parameters, wherein the storage IO operation performs at least one in-line data compression operations on at least a portion of the data group, the at least one in-line data compression operations comprising compressing or decompressing at least the portion of the data group according to the compression parameters prior to returning or writing the one or more data units at one or more storage locations specified by the at least one storage IO operation; and
returning a status to the application, the status pertaining to the execution of the storage IO operation.
20. The system of claim 19 , wherein the at least one in-line data compression operation completes before returning the status pertaining to the execution of the storage IO operation to the application.