Heat indices for file systems and block storage
View Patent ↗Techniques and mechanisms are provided to allow for selective optimization, including deduplication and/or compression, of portions of files and data blocks. Data access is monitored to generate a heat index for identifying sections of files and volumes that are frequently and infrequently accessed. These frequently used portions may be left non-optimized to reduce or eliminate optimization I/O overhead. Infrequently accessed portions can be more aggressively optimized.
1. A method, comprising:
determining whether a first portion of a first file is less frequently accessed than a second portion of the first file;
based on a determination that the first portion of the first file is less frequently accessed than the second portion of the first file, performing optimization on the first portion of the first file more aggressively to achieve a higher reduction ratio on the first portion of the first file than on the second portion of the first file, wherein optimization comprises deduplication of data in the first file.
2. The method of claim 1 , wherein access comprises processor read and write access.
3. The method of claim 1 , further comprising monitoring access to a plurality of portions comprises maintaining access time information.
4. The method of claim 1 , further comprising maintaining a heat index.
5. The method of claim 1 , wherein optimization further comprises compression.
6. The method of claim 1 , wherein the second portion of the first file is left unoptimized.
7. The method of claim 1 , wherein a third portion of the first file is accessed more frequently than the second portion of the first file.
8. The method of claim 1 , wherein an optimization scale is applied to determine the aggressiveness of optimization based on the frequency of access.
9. The method of claim 1 , wherein a heat index is used to determine the aggressiveness of optimization based on the frequency of access.
10. The method of claim 1 , wherein more aggressive optimization results in smaller file sizes.
11. An apparatus, comprising:
a processor configured determine whether a first portion of a first file is less frequently accessed than a second portion of the first file;
wherein the processor is further configured to optimize the first file;
wherein optimization is performed on the first portion of the first file more aggressively based on a determination that the first portion of the first file is less frequently accessed than the second portion of the first file to achieve a higher reduction ratio on the first portion of the first file than on the second portion of the first file, wherein optimization comprises deduplication of data in the first file.
12. The apparatus of claim 11 , wherein access comprises processor read and write access.
13. The apparatus of claim 11 , wherein the processor is further configured to monitor access to a plurality of portions comprises maintaining access time information.
14. The apparatus of claim 11 , wherein the processor is further configured to maintain a heat index.
15. The apparatus of claim 11 , wherein optimization further comprises compression.
16. The apparatus of claim 11 , wherein the second portion of the first file is left unoptimized.
17. The apparatus of claim 11 , wherein a third portion of the first file is accessed more frequently than the second portion of the first file.
18. The apparatus of claim 11 , wherein an optimization scale is applied to determine the aggressiveness of optimization based on the frequency of access.
19. The apparatus of claim 11 , wherein a heat index is used to determine the aggressiveness of optimization based on the frequency of access.
20. A non-transitory computer readable medium, comprising:
computer code for determining whether a first portion of a first file is less frequently accessed than a second portion of the first file; and
computer code for optimizing the first file;
wherein optimization is performed on the first portion of the first file more aggressively based on a determination that the first portion of the first file is less frequently accessed than the second portion of the first file to achieve a higher reduction ratio on the first portion of the first file than on the second portion of the first file, wherein optimization comprises deduplication of data in the first file.