Method and apparatus for servicing read and write requests using a cache replacement catalog
Methods and systems to intelligently cache content in a virtualization environment using virtualization software such as VMWare ESX or Citrix XenServer or Microsoft HyperV or Redhat KVM or their variants are disclosed. Storage IO operations (reads from and writes to disk) are analyzed (or characterized) for their overall value and pinned to cache if their value exceeds a certain defined threshold based on criteria specific to the New Technology File System (NTFS) file-system. Analysis/characterization of NTFS file systems for intelligent dynamic caching include analyzing storage block data associated with a Virtual Machine of interest in accordance with a pre-determined data model to determine the value of the block under analysis for long term or short term caching. Integer values assigned to different types of NTFS objects in a white list data structure called a catalog that can be used to analyze the storage block data.
1. A method, comprising:
reading into memory a catalog file including a hash table having a plurality of entries for new technology file system (NTFS) objects and a plurality of associated values, each associated value comprises an integer, and each entry is assigned only one of the plurality of associated values;
waiting to service block read requests and block write requests;
initializing an intelligent content aware caching application running on a VM hypervisor only after a block read request has been received;
populating a slot in a cache with a payload and metadata from each block read request or from each block write request that is received after the caching application has been initialized;
updating the metadata in each slot with one of the plurality of assigned values from the catalog according to a read input-output (IO) protocol for block read requests or according to a write IO protocol for block write requests, wherein the caching application updates each slot using the plurality of entries for NTFS objects and the plurality of associated values read into memory from the catalog.
2. The method of claim 1 , wherein the read IO protocol comprises
hashing contents of a read IO block using a hashing function to generate a hash value;
storing the hash value in memory;
comparing the hash value against the plurality of entries for NTFS objects in the catalog;
determining if the hash value exists as one of plurality of one of the plurality of assigned values in the catalog;
updating metadata in the slot for the read IO block with an assigned value of zero from the catalog if the hash value does not exist as one of the plurality of assigned values;
foregoing an update of metadata in the slot for the read IO block with an assigned value from the catalog if the hash value exists as one of the plurality of assigned values and the assigned value is a one; and
updating metadata in the slot for the read IO block with an assigned value from the catalog if the hash value exists as one of the plurality of assigned values but the assigned value is not a one.
3. The method of claim 1 , wherein the write IO protocol comprises
hashing contents of a write IO block using a hashing function to generate a hash value;
storing the hash value in memory;
comparing the hash value against the plurality of entries for NTFS objects in the catalog;
determining if the hash value exists as one of plurality of one of the plurality of assigned values in the catalog;
updating metadata in the slot for the write IO block with an assigned value of zero from the catalog if the hash value does not exist as one of the plurality of assigned values;
foregoing an update of metadata in the slot for the write IO block with an assigned value from the catalog if the hash value exists as one of the plurality of assigned values and the assigned value is a one; and
updating metadata in the slot for the write IO block with an assigned value from the catalog if the hash value exists as one of the plurality of assigned values but the assigned value is not a one.
4. The method of claim 1 , wherein each associated value comprises a score indicative of a relative importance of corresponding NTFS objects.
5. A non-transitory computer readable storage medium including instructions that, when executed on a computer system cause the computer system to perform a method comprising:
reading into memory a catalog file including a hash table having a plurality of entries for new technology file system (NTFS) objects and a plurality of associated values, each associated value comprises an integer, and each entry is assigned only one of the plurality of associated values;
waiting to service block read requests and block write requests;
initializing an intelligent content aware caching application running on a VM hypervisor only after a block read request has been received;
populating a slot in a cache with a payload and metadata from each block read request or from each block write request that is received after the caching application has been initialized;
updating the metadata in each slot with one of the plurality of assigned values from the catalog according to a read input-output (IO protocol for block read requests or according to a write IO protocol for block write requests, wherein the caching application updates each slot using the plurality of entries for NTFS objects and the plurality of associated values read into memory from the catalog.
6. The non-transitory computer readable storage medium of claim 5 further comprising read IO protocol instructions that, when executed on the computer system cause the computer system to perform the method comprising:
hashing contents of a read IO block using a hashing function to generate a hash value;
storing the hash value in memory;
comparing the hash value against the plurality of entries for NTFS objects in the catalog;
determining if the hash value exists as one of plurality of one of the plurality of assigned values in the catalog;
updating metadata in the slot for the read IO block with an assigned value of zero from the catalog if the hash value does not exist as one of the plurality of assigned values;
foregoing an update of metadata in the slot for the read IO block with an assigned value from the catalog if the hash value exists as one of the plurality of assigned values and the assigned value is a one; and
updating metadata in the slot for the read IO block with an assigned value from the catalog if the hash value exists as one of the plurality of assigned values but the assigned value is not a one.
7. The non-transitory computer readable storage medium of claim 5 further comprising write IO protocol instructions that, when executed on the computer system cause the computer system to perform the method comprising:
hashing contents of a write IO block using a hashing function to generate a hash value;
storing the hash value in memory;
comparing the hash value against the plurality of entries for NTFS objects in the catalog;
determining if the hash value exists as one of plurality of one of the plurality of assigned values in the catalog;
updating metadata in the slot for the write IO block with an assigned value of zero from the catalog if the hash value does not exist as one of the plurality of assigned values;
foregoing an update of metadata in the slot for the write IO block with an assigned value from the catalog if the hash value exists as one of the plurality of assigned values and the assigned value is a one; and
updating metadata in the slot for the write IO block with an assigned value from the catalog if the hash value exists as one of the plurality of assigned values but the assigned value is not a one.
8. The non-transitory computer readable storage medium of claim 5 , wherein each associated value comprises a score indicative of a relative importance of corresponding NTFS objects.
9. A system, comprising:
a memory; and
a processor coupled with the memory configured to
read into the memory a catalog file including a hash table having a plurality of entries for new technology file system (NTFS) objects and a plurality of associated values, each associated value comprises an integer, and each entry is assigned only one of the plurality of associated values,
wait to service block read requests and block write requests,
initialize an intelligent content aware caching application running on a VM hypervisor only after a block read request has been received,
populate a slot in a cache with a payload and metadata from each block read request or from each block write request that is received after the caching application has been initialized,
update the metadata in each slot with one of the plurality of assigned values from the catalog according to a read input-output (IO) protocol for block read requests or according to a write input-output (IO) protocol for block write requests, wherein the caching application updates each slot using the plurality of entries for NTFS objects and the plurality of associated values read into memory from the catalog.
10. The system of claim 9 , wherein the processor configured to execute the read IO protocol further comprising the processor to
hash contents of a read IO block using a hashing function to generate a hash value,
storing the hash value in memory,
compare the hash value against the plurality of entries for NTFS objects in the catalog,
determine if the hash value exists as one of plurality of one of the plurality of assigned values in the catalog,
update metadata in the slot for the read IO block with an assigned value of zero from the catalog if the hash value does not exist as one of the plurality of assigned values,
forego an update of metadata in the slot for the read IO block with an assigned value from the catalog if the hash value exists as one of the plurality of assigned values and the assigned value is a one, and
update metadata in the slot for the read IO block with an assigned value from the catalog if the hash value exists as one of the plurality of assigned values but the assigned value is not a one.
11. The system of claim 9 , wherein the processor configured to execute the write IO protocol further comprising the processor to
hash contents of a write IO block using a hashing function to generate a hash value,
store the hash value in memory,
compare the hash value against the plurality of entries for NTFS objects in the catalog,
determine if the hash value exists as one of plurality of one of the plurality of assigned values in the catalog
update metadata in the slot for the write IO block with an assigned value of zero from the catalog if the hash value does not exist as one of the plurality of assigned values,
forego an update of metadata in the slot for the write IO block with an assigned value from the catalog if the hash value exists as one of the plurality of assigned values and the assigned value is a one, and
update metadata in the slot for the write IO block with an assigned value from the catalog if the hash value exists as one of the plurality of assigned values but the assigned value is not a one.
12. The system of claim 9 , wherein each associated value comprises a score indicative of a relative importance of corresponding NTFS objects.