Reduced downtime for a virtual machine content-based read cache
A system is described for enabling content-based read caching on a virtual machine without downtime for creating the digest file by launching the virtual machine before the digest file is produced and creating the digest file dynamically during runtime. After the virtual machine is executed, a background process creates the digest file in the background. The background process is paused during input/output operations and resumes when there are no pending input/output requests.
1. A method for caching data in a virtual machine (VM), comprising:
executing the virtual machine;
after the virtual machine is executed, initiating a background process for creating a digest file corresponding to a virtual disk of the virtual machine, the background process comprising:
partitioning the virtual disk into a plurality of blocks;
generating a key for each block and storing the key in the digest file;
receiving a request to read a first block from the virtual disk;
in response to the request to read the first block:
locating a key corresponding to the first block in the digest file;
locating the first block in a cache of the VM by comparing the key corresponding to the first block in the digest file with keys of blocks located in the cache;
retrieving the first block from the cache to service the read request;
receiving a request to write a second block to the virtual disk;
in response to the request to write the second block to the virtual disk, pausing the background process for creating the digest file and, while the background process for creating the digest file is paused, writing the second block to the virtual disk to service the request;
calculating a key for the second block; and
storing the calculated key for the second block in the digest file.
2. The method of claim 1 , further comprising:
receiving a request to read a third block from the virtual disk;
determining that a key corresponding to the third block is not available in the digest file;
pausing the background process for creating the digest file;
retrieving the third block from the virtual disk to service the read request;
calculating a key for the third block;
storing the calculated key for the third block in the digest file; and
resuming the background process for creating the digest file.
3. The method of claim 1 , further comprising:
determining that no read or write requests are pending; and
in response to determining that no read or write requests are pending, resuming the background process for creating the digest file.
4. The method of claim 1 , further comprising:
partitioning the virtual disk into a plurality of sequential ranges, each sequential range comprising a sequence of multiple blocks of the virtual disk;
by the background process for creating the digest file:
processing the sequential ranges of the virtual disk by generating a key for each block in a sequential range and storing the key in the digest file;
after a key for each block in the sequential range is produced and stored in the digest file, setting a corresponding value in a data structure for tracking the creation of the digest file to indicate that the sequential range is completed in the digest file, the data structure containing a corresponding value for each sequential range of the virtual disk to indicate whether the corresponding sequential range has been completed in the digest file; and
wherein the background process for creating the digest file checks the data structure to determine which sequential ranges of the virtual disk to process to produce the digest file.
5. The method of claim 4 , further comprising:
receiving a request to read a third block from the virtual disk;
determining that a key corresponding to the third block is not available in the digest file;
pausing the background process for creating the digest file;
retrieving the third block to service the read request;
calculating a key for the third block and for each other block in the sequential range of the virtual disk in which the third bock is located;
storing the calculated keys in the digest file;
setting a corresponding value in the data structure for tracking the creation of the digest file to indicate that the sequential range where the third block is located has been completed in the digest file; and
resuming the background process for creating the digest file.
6. The method of claim 4 , further comprising:
receiving a request to write a third block to the virtual disk;
pausing the background process for creating the digest file;
calculating a key for the third block and for each other block in the sequential range of the virtual disk in which the third bock will be located;
storing the calculated keys in the digest file;
writing the third block to the virtual disk to service the request;
setting a corresponding value in the data structure for tracking the creation of the digest file to indicate that the sequential range where the third block is located has been completed in the digest file; and
resuming the background process for creating the digest file.
7. The method of claim 1 , further comprising:
by the background process:
checking a queue for storing received read and write requests;
determining that a read or write request is present in the queue;
in response to determining that the read or write request is present in the queue, signaling a high priority thread to process the read or write request in the queue and going to sleep;
receiving a signal to wake up and resume creating the digest file, the signal being sent in response to no more requests being present in the queue.
8. A computing device for caching data in a virtual machine (VM), comprising:
at least one processor; and
memory including instructions that, when executed by the at least one processor, cause the computing device to perform the steps of:
executing the virtual machine;
after the virtual machine is executed, initiating a background process for creating a digest file corresponding to a virtual disk of the virtual machine, the background process comprising:
partitioning the virtual disk into a plurality of blocks;
generating a key for each block and storing the key in the digest file;
receiving a request to read a first block from the virtual disk;
in response to the request to read the first block:
locating a key corresponding to the first block in the digest file;
locating the first block in a cache of the VM by comparing the key corresponding to the first block in the digest file with keys of blocks located in the cache;
retrieving the first block from the cache to service the read request;
receiving a request to write a second block to the virtual disk;
in response to the request to write the second block to the virtual disk, pausing the background process for creating the digest file and, while the background process for creating the digest file is paused, writing the second block to the virtual disk to service the request;
calculating a key for the second block; and
storing the calculated key for the second block in the digest file.
9. The computing device of claim 8 , wherein the memory further includes instructions that when executed by the at least one processor, cause the computing device to perform the steps of:
receiving a request to read a third block from the virtual disk;
determining that a key corresponding to the third block is not available in the digest file;
pausing the background process for creating the digest file;
retrieving the third block from the virtual disk to service the read request;
calculating a key for the third block;
storing the calculated key for the third block in the digest file; and
resuming the background process for creating the digest file.
10. The computing device of claim 8 , wherein the memory further includes instructions that when executed by the at least one processor, cause the computing device to perform the steps of:
determining that no read or write requests are pending; and
in response to determining that no read or write requests are pending, resuming the background process for creating the digest file.
11. The computing device of claim 8 , wherein the memory further includes instructions that when executed by the at least one processor, cause the computing device to perform the steps of:
partitioning the virtual disk into a plurality of sequential ranges, each sequential range comprising a sequence of multiple blocks of the virtual disk;
by the background process for creating the digest file:
processing the sequential ranges of the virtual disk by generating a key for each block in a sequential range and storing the key in the digest file;
after a key for each block in the sequential range is produced and stored in the digest file, setting a corresponding value in a data structure for tracking the creation of the digest file to indicate that the sequential range is completed in the digest file, the data structure containing a corresponding value for each sequential range of the virtual disk to indicate whether the corresponding sequential range has been completed in the digest file; and
wherein the background process for creating the digest file checks the data structure to determine which sequential ranges of the virtual disk to process to produce the digest file.
12. The computing device of claim 11 , wherein the memory further includes instructions that when executed by the at least one processor, cause the computing device to perform the steps of:
receiving a request to read a third block from the virtual disk;
determining that a key corresponding to the third block is not available in the digest file;
pausing the background process for creating the digest file;
retrieving the third block to service the read request;
calculating a key for the third block and for each other block in the sequential range of the virtual disk in which the third bock is located;
storing the calculated keys in the digest file;
setting a corresponding value in the data structure for tracking the creation of the digest file to indicate that the sequential range where the third block is located has been completed in the digest file; and
resuming the background process for creating the digest file.
13. The computing device of claim 11 , wherein the memory further includes instructions that when executed by the at least one processor, cause the computing device to perform the steps of:
receiving a request to write a third block to the virtual disk;
pausing the background process for creating the digest file;
calculating a key for the third block and for each other block in the sequential range of the virtual disk in which the third block will be located;
storing the calculated keys in the digest file;
writing the third block to the virtual disk to service the request;
setting a corresponding value in the data structure for tracking the creation of the digest file to indicate that the sequential range where the third block is located has been completed in the digest file; and
resuming the background process for creating the digest file.
14. The computing device of claim 8 , wherein the memory further includes instructions that when executed by the at least one processor, cause the computing device to perform the steps of:
by the background process:
checking a queue for storing received read and write requests;
determining that a read or write request is present in the queue;
in response to determining that the read or write request is present in the queue, signaling a high priority thread to process the read or write request in the queue and going to sleep;
receiving a signal to wake up and resume creating the digest file, the signal being sent in response to no more requests being present in the queue.
15. A non-transitory computer readable storage medium for caching data in a virtual machine (VM), comprising one or more sequences of instructions, the instructions when executed by one or more processors causing the one or more processors to execute the operations of:
executing the virtual machine;
after the virtual machine is executed, initiating a background process for creating a digest file corresponding to a virtual disk of the virtual machine, the background process comprising:
partitioning the virtual disk into a plurality of blocks;
generating a key for each block and storing the key in the digest file;
receiving a request to read a first block from the virtual disk;
in response to the request to read the first block:
locating a key corresponding to the first block in the digest file;
locating the first block in a cache of the VM by comparing the key corresponding to the first block in the digest file with keys of blocks located in the cache;
retrieving the first block from the cache to service the read request;
receiving a request to write a second block to the virtual disk;
in response to the request to write the second block to the virtual disk, pausing the background process for creating the digest file and, while the background process for creating the digest file is paused, writing the second block to the virtual disk to service the request;
calculating a key for the second block; and
storing the calculated key for the second block in the digest file.
16. The non-transitory computer readable storage medium of claim 15 , further comprising instructions that when executed by the one or more processors cause the one or more processors to execute the operations of:
receiving a request to read a third block from the virtual disk;
determining that a key corresponding to the third block is not available in the digest file;
pausing the background process for creating the digest file;
retrieving the third block from the virtual disk to service the read request;
calculating a key for the third block;
storing the calculated key for the third block in the digest file; and
resuming the background process for creating the digest file.
17. The non-transitory computer readable storage medium of claim 15 , further comprising instructions that when executed by the one or more processors cause the one or more processors to execute the operations of:
determining that no read or write requests are pending; and
in response to determining that no read or write requests are pending, resuming the background process for creating the digest file.
18. The non-transitory computer readable storage medium of claim 15 , further comprising instructions that when executed by the one or more processors cause the one or more processors to execute the operations of:
partitioning the virtual disk into a plurality of sequential ranges, each sequential range comprising a sequence of multiple blocks of the virtual disk;
by the background process for creating the digest file:
processing the sequential ranges of the virtual disk by generating a key for each block in a sequential range and storing the key in the digest file;
after a key for each block in the sequential range is produced and stored in the digest file, setting a corresponding value in a data structure for tracking the creation of the digest file to indicate that the sequential range is completed in the digest file, the data structure containing a corresponding value for each sequential range of the virtual disk to indicate whether the corresponding sequential range has been completed in the digest file; and
wherein the background process for creating the digest file checks the data structure to determine which sequential ranges of the virtual disk to process to produce the digest file.
19. The non-transitory computer readable storage medium of claim 18 , further comprising instructions that when executed by the one or more processors cause the one or more processors to execute the operations of:
receiving a request to read a third block from the virtual disk;
determining that a key corresponding to the third block is not available in the digest file;
pausing the background process for creating the digest file;
retrieving the third block to service the read request;
calculating a key for the third block and for each other block in the sequential range of the virtual disk in which the third bock is located;
storing the calculated keys in the digest file;
setting a corresponding value in the data structure for tracking the creation of the digest file to indicate that the sequential range where the third block is located has been completed in the digest file; and
resuming the background process for creating the digest file.
20. The non-transitory computer readable storage medium of claim 18 , further comprising instructions that when executed by the one or more processors cause the one or more processors to execute the operations of:
receiving a request to write a third block to the virtual disk;
pausing the background process for creating the digest file;
calculating a key for the third block and for each other block in the sequential range of the virtual disk in which the third block will be located;
storing the calculated keys in the digest file;
writing the third block to the virtual disk to service the request;
setting a corresponding value in the data structure for tracking the creation of the digest file to indicate that the sequential range where the third block is located has been completed in the digest file; and
resuming the background process for creating the digest file.