System-directed checkpointing implementation using a hypervisor layer
While system-directed checkpointing can be implemented in various ways, for example by adding checkpointing support in the memory controller or in the operating system in otherwise standard computers, implementation at the hypervisor level enables the necessary state information to be captured efficiently while providing a number of ancillary advantages over those prior-art methods. This disclosure details procedures for realizing those advantages through relatively minor modifications to normal hypervisor operations. Specifically, by capturing state information in a guest-operating-system-specific manner, any guest operating system can be rolled back independently and resumed without losing either program or input/output (I/O) continuity and without affecting the operation of the other operating systems or their associated applications supported by the same hypervisor. Similarly, by managing I/O queues as described in this disclosure, rollback can be accomplished without requiring I/O operations to be repeated and I/O device failures can be circumvented without losing any I/O data in the process.
1. A computer system configured to run a hypervisor operable to enable the state of one or more guest operating systems to be checkpointed, comprising:
a memory-manage subsystem configured to maintain and use a temporary-read-only bit in tables used to convert pseudo-physical addresses of a hypervisor's guest operating system (OS) to physical addresses, and to use the temporary read-only bit in conjunction with another read-only bit to trigger a trap by which normal read-only page violations and violations for checkpointing are distinguishable by the hypervisor;
a device-emulator subsystem configured to maintain input/output (I/O) read- and write-buffers in the hypervisor's virtual space, whose contents are preserved until the guest OS is checkpointed following completion of the I/O operation; and
a rollback and recovery subsystem configured to periodically checkpoint the state of each guest OS of the hypervisor.
2. The system of claim 1 in which the memory-management subsystem maintains the temporary-read-only bit in the directory addresses and page maps used to convert the guest OS's pseudo-physical addresses to physical addresses.
3. The system of claim 1 in which the input/output (I/O) read- and write-buffers maintained in the hypervisor's virtual space are used for all I/O operations, and wherein preserving the contents until the guest operating system is checkpointed following the completion of the I/O operation enables I/O operations to continue uninterrupted using standard, unmodified I/O drivers.
4. The system of claim 3 , wherein the device-emulator subsystem is configured to manage three lists of input/output (I/O) request blocks for each guest operating system (OS) comprising:
a list of those requests that have been made by the guest OS but have not yet been completed;
a list of those requests that have been completed since the guest OS was last checkpointed;
a list of those requests that were acknowledged to the guest OS as having been completed at the time of its last checkpoint.
5. The system of claim 3 in which each emulator maintains device-specific lists recording the state of the subject device following the execution of each pending input and output request and uses the relevant list when a device encounters a fault and needs to be reset to the state that prevailed following the execution of the last completed request.
6. The system of claim 1 , wherein the checkpointing of the state of each guest OS permits that, in the event of a fault, any affected guest OS can be individually rolled back to its last checkpointed state and restarted.
7. The system of claim 6 , wherein the rollback and recovery subsystem is so configured that any guest OS can be rolled back to its last checkpointed state, independently of all other guest operating systems running on the same computer system.
8. The system of claim 6 , wherein the rollback and recovery subsystem is so configured that ongoing I/O operations are not interrupted in the event of a fault and need not be repeated following any such rollback.
9. The system of claim 1 in which the hypervisor itself can be checkpointed and rolled back, while still providing the option of rolling back each guest operating system individually.