Method and apparatus for preparing a cache replacement catalog
Methods and systems to intelligently cache content in a virtualization environment using virtualization software such as VMWare ESX or Citrix XenServer or Microsoft HyperV or Redhat KVM or their variants are disclosed. Storage IO operations (reads from and writes to disk) are analyzed (or characterized) for their overall value and pinned to cache if their value exceeds a certain defined threshold based on criteria specific to the New Technology File System (NTFS) file-system. Analysis/characterization of NTFS file systems for intelligent dynamic caching include analyzing storage block data associated with a Virtual Machine of interest in accordance with a pre-determined data model to determine the value of the block under analysis for long term or short term caching. Integer values assigned to different types of NTFS objects in a white list data structure called a catalog that can be used to analyze the storage block data.
1. A method, comprising:
creating a virtual machine (VM) image of an operating system (OS);
loading the VM image into a caching application using a mount utility;
enumerating file contents of the OS and program file directories on a root file system;
storing the enumeration as a directory enumeration result;
reading in sequence, from beginning to end, each file in the directory enumeration result in a predetermined segment size;
hashing each segment read using a hashing function to generate a hash entry for each segment, the hash entry including an assigned value that is an integer associated with hash entry based on the file contents; and
storing each hash entry and its associated assigned value in a hash table data structure.
2. The method of claim 1 , wherein the predetermined segment size comprises a NTFS cluster size.
3. The method of claim 2 , wherein the NTFS cluster size is 4 kilobytes.
4. The method of claim 1 , wherein the OS comprises a Windows OS.
5. The method of claim 1 , wherein each assigned value comprises a score indicative of a relative importance of each segment of file contents.
6. The method of claim 5 , wherein the score for segments are evaluated during cache replacement to determine corresponding segments to evict from a cache memory.
7. A non-transitory computer readable storage medium including instructions that, when executed on a computer system cause the computer system to perform a method comprising:
creating a virtual machine (VM) image of an operating system (OS);
loading the VM image into a caching application using a mount utility;
enumerating file contents of the OS and program file directories on a root file system;
storing the enumeration as a directory enumeration result;
reading in sequence, from beginning to end, each file in the directory enumeration result in a predetermined segment size;
hashing each segment read using a hashing function to generate a hash entry for each segment, the hash entry including an assigned value that is an integer associated with hash entry based on the file contents; and
storing each hash entry and its associated assigned value in a hash table data structure.
8. The non-transitory computer readable storage medium of claim 7 , wherein the predetermined segment size comprises a NTFS cluster size.
9. The non-transitory computer readable storage medium of claim 8 , wherein the NTFS cluster size is 4 kilobytes.
10. The non-transitory computer readable storage medium of claim 7 , wherein the OS comprises a Windows OS.
11. The non-transitory computer readable storage medium of claim 7 , wherein each assigned value comprises a score indicative of a relative importance of each segment of file contents.
12. The non-transitory computer readable storage medium of claim 11 , wherein the score for segments are evaluated during cache replacement to determine corresponding segments to evict from a cache memory.
13. A system, comprising:
a memory to store a hash data table structure; and
a processor coupled with the memory to
create a virtual machine (VM) image of an operating system (OS),
load the VM image into a caching application using a mount utility,
enumerate file contents of the OS and program file directories on a root file system,
store the enumeration as a directory enumeration result,
read in sequence, from beginning to end, each file in the directory enumeration result in a predetermined segment size,
hash each segment read using a hashing function to generate a hash entry for each segment, the hash entry including an assigned value that is an integer associated with hash entry based on the file contents, and
store each hash entry and its associated assigned value in the hash table data structure in the memory.
14. The system of claim 13 , wherein the predetermined segment size comprises a NTFS cluster size.
15. The system of claim 14 , wherein the NTFS cluster size is 4 kilobytes.
16. The system of claim 13 , wherein the OS comprises a Windows OS.
17. The system of claim 13 , wherein each assigned value comprises a score indicative of a relative importance of each segment of file contents.
18. The system of claim 13 , wherein the score for segments are evaluated during cache replacement to determine corresponding segments to evict from a cache memory.