Managing inline data compression in storage systems
A method is used in managing inline data compression in storage systems. A number of allocation units in which data of a file system is stored in a segment in a compressed format is determined. The segment comprises of a first storage space for storing a set of allocation units and a second storage space that is not allocated to the set of allocation units. Based on a policy, the second storage space of the segment is associated to at least one allocation unit of the set of allocation units.
1. A method for use in managing inline data compression in storage systems, the method comprising:
determining a number of allocation units in which data of a file system is stored in a segment in a compressed format;
determining a size of a first storage space in the segment for storing a set of allocation units and a size of a second storage space in the segment that is not allocated to the set of allocation units, wherein the second storage space within the segment; and
based on a policy and the size of the second storage space, increasing a size of at least one allocation unit of the set of allocation units by at least one of: adding the unused storage space to a last compressed allocation unit within the segment or distributing the unused storage space across the set of allocation units in the segment to increase a probability that data associated with subsequent write I/O requests to the file system can be accommodated by the segment.
2. The method of claim 1 , wherein the number of allocation units is determined based on statistical analysis by applying predictive techniques.
3. The method of claim 1 , wherein the policy includes increasing the last allocation unit of the segment by the size of the second storage space.
4. The method of claim 1 , wherein the policy includes increasing each allocation unit of the segment based on the size of the second storage space and the number of allocation units.
5. The method of claim 1 , further comprising:
evaluating information for determining the number of allocation units in which the segment is apportioned, wherein evaluating the information includes determining for different sets of file system blocks an amount of storage space used by the different sets of file system blocks, the size of a compressed segment including the different sets of file system blocks, and the number of allocation units allocated in the segment.
6. The method of claim 5 , further comprising:
based on the evaluation, determining the amount of storage space that remains unused at the end of a compressed segment.
7. The method of claim 1 , wherein the file system is organized as a file system hierarchy of file system blocks, wherein the file system blocks include a set of indirect data blocks, each indirect data block including a set of data blocks.
8. The method of claim 1 , wherein a persistent file data cache aggregation logic selects a set of allocation units for compressing the set of allocation units in a segment, wherein an allocation unit is a file system block.
9. The method of claim 1 , wherein a mapping pointer is associated with a file system block of a file, wherein the mapping pointer includes a distributed weight indicating whether the file system block has been shared.
10. The method of claim 9 , wherein a mapping pointer of a file system block refers to a segment virtual block pointer, wherein the segment virtual block pointer includes an extent list including information regarding a set of allocation units compressed in a segment, wherein a file system block number included in the segment virtual block pointer refers to the segment, wherein the segment includes a set of contiguous file system data blocks in a compressed format.
11. A system for use in managing inline data compression in storage systems, the system comprising a processor configured to:
determine a number of allocation units in which data of a file system is stored in a segment in a compressed format;
determine a size of a first storage space in the segment for storing a set of allocation units and a size of a second storage space in the segment that is not allocated to the set of allocation units, wherein the second storage space is unused storage space within the segment; and
based on a policy and the size of the second storage space, increase a size of at least one allocation unit of the set of allocation units by at least one of: adding the unused storage space to a last compressed allocation unit within the segment or distributing the unused storage space across the set of allocation units in the segment to increase a probability that data associated with subsequent write I/O requests to the file system can be accommodated by the segment.
12. The system of claim 11 , wherein the number of allocation units is determined based on statistical analysis by applying predictive techniques.
13. The system of claim 11 , wherein the policy includes increasing the last allocation unit of the segment by the size of the second storage space.
14. The system of claim 11 , wherein the policy includes increasing each allocation unit of the segment based on the size of the second storage space and the number of allocation units.
15. The system of claim 11 , further comprising:
evaluate information for determining the number of allocation units in which the segment is apportioned, wherein evaluating the information includes determining for different sets of file system blocks an amount of storage space used by the different sets of file system blocks, the size of a compressed segment including the different sets of file system blocks, and the number of allocation units allocated in the segment.
16. The system of claim 15 , further comprising:
based on the evaluation, determine the amount of storage space that remains unused at the end of a compressed segment.
17. The system of claim 11 , wherein the file system is organized as a file system hierarchy of file system blocks, wherein the file system blocks include a set of indirect data blocks, each indirect data block including a set of data blocks.
18. The system of claim 11 , wherein a persistent file data cache aggregation logic selects a set of allocation units for compressing the set of allocation units in a segment, wherein an allocation unit is a file system block.
19. The system of claim 11 , wherein a mapping pointer is associated with a file system block of a file, wherein the mapping pointer includes a distributed weight indicating whether the file system block has been shared.
20. The system of claim 19 , wherein a mapping pointer of a file system block refers to a segment virtual block pointer, wherein the segment virtual block pointer includes an extent list including information regarding a set of allocation units compressed in a segment, wherein a file system block number included in the segment virtual block pointer refers to the segment, wherein the segment includes a set of contiguous file system data blocks in a compressed format.