Heat indices for file systems and block storage
View Patent ↗Techniques and mechanisms are provided to allow for selective optimization, including deduplication and/or compression, of portions of files and data blocks. Data access is monitored to generate a heat index for identifying sections of files and volumes that are frequently and infrequently accessed. These frequently used portions may be left non-optimized to reduce or eliminate optimization I/O overhead. Infrequently accessed portions can be more aggressively optimized.
1. A method, comprising:
determining whether a first portion of a file is less frequently accessed than a second portion of the file;
based on a determination that the first portion of the file is less frequently accessed than the second portion of the file, performing optimization using a higher reduction ratio on the first portion of the file than on the second portion of the file.
2. The method of claim 1 , wherein access comprises processor read and write access.
3. The method of claim 1 , further comprising monitoring access to a plurality of portions comprises maintaining access time information.
4. The method of claim 1 , further comprising maintaining a heat index.
5. The method of claim 1 , wherein optimization comprises compression.
6. The method of claim 1 , wherein the second portion of the file is left unoptimized.
7. The method of claim 1 , wherein a third portion of the file is accessed more frequently than the second portion of the file.
8. The method of claim 1 , wherein an optimization scale is applied to determine reduction rate during optimization based on the frequency of access.
9. The method of claim 1 , wherein a heat index is used to determine reduction rate during optimization based on the frequency of access.
10. The method of claim 1 , wherein optimization comprises deduplication or compression.
11. An apparatus, comprising:
a processor configured determine whether a first portion of a file is less frequently accessed than a second portion of the file;
wherein the processor is further configured to optimize the file; wherein optimization is performed using a higher reduction ratio on the first portion of the file than on the second portion of the file based on a determination that the first portion of the file is less frequently accessed than the second portion of the file.
12. The apparatus of claim 11 , wherein access comprises processor read and write access.
13. The apparatus of claim 11 , wherein the processor is further configured to monitor access to a plurality of portions comprises maintaining access time information.
14. The apparatus of claim 11 , wherein the processor is further configured to maintain a heat index.
15. The apparatus of claim 11 , wherein optimization comprises deduplication or compression.
16. The apparatus of claim 11 , wherein the second portion of the file is left unoptimized.
17. The apparatus of claim 11 , wherein a third portion of the file is accessed more frequently than the second portion of the file.
18. The apparatus of claim 11 , wherein an optimization scale is applied to determine reduction rate during optimization based on the frequency of access.
19. The apparatus of claim 11 , wherein a heat index is used to determine reduction rate during optimization based on the frequency of access.
20. A non-transitory computer readable medium, comprising:
computer code for determining whether a first portion of a file is less frequently accessed than a second portion of the file; and
computer code for optimizing the file;
wherein optimization is performed using a higher reduction ratio on the first portion of the file than on the second portion of the file based on a determination that the first portion of the file is less frequently accessed than the second portion of the file.