Asynchronous in-memory data backup system
Data backup systems and methods are provided to asynchronously mirror in-memory checkpoint images generated by a process executing on a host system. For example, a data backup system is configured to: receive a checkpoint request message to store a copy of a checkpoint image which includes memory pages residing in a first memory region of memory on the host system; pull a copy of metadata associated with the memory pages from a second memory region of memory on the host system; pull a copy of the memory pages of the checkpoint image from the first memory region; generate a copy of the checkpoint image using the memory pages and associated metadata pulled from the host system; and store the copy of the checkpoint image in a storage device with an in-memory format that is the same as the checkpoint image residing in the first memory region on the host system.
1. A method, comprising:
receiving, by a data backup system, notification of a checkpoint request issued by a process executing on a host computing system to store a copy of a checkpoint image comprising one or more memory pages that reside in a first memory region of memory on the host computing system;
pulling, by the data backup system, a copy of metadata from a metadata log residing in a second memory region of memory on the host computing system, wherein the metadata is associated with the one or more memory pages of the checkpoint image;
pulling, by the data backup system, a copy of the one or more memory pages of the checkpoint image from the first memory region of memory on the host computing system;
generating, by the data backup system, a copy of the checkpoint image using the copies of the one or more memory pages and the associated metadata pulled from the host computing system; and
storing, by the data backup system, the copy of the checkpoint image in a storage device on the data backup system;
wherein the copy of the checkpoint image stored in the storage device on the data backup system comprises a same in-memory format as the checkpoint image residing in the first memory region of memory on the host computing system.
2. The method of claim 1 , wherein pulling a copy of the metadata from the metadata log residing in the second memory region of memory on the host computing system comprises performing, by the data backup system, at least one remote direct memory access (RDMA) read operation to transfer the copy of the metadata from a non-volatile random access memory device on the host computing system to a non-volatile random access memory device on the data backup system.
3. The method of claim 2 , wherein pulling a copy of the one or more memory pages of the checkpoint image from the first memory region of memory on the host computing system comprises performing, by the data backup system, one or more RDMA read operations to transfer the copy of the memory pages from a RDMA-enabled non-volatile memory on the host computing system to the non-volatile random access memory device on the data backup system.
4. The method of claim 1 , wherein the one or more memory pages of the checkpoint image comprises modified memory pages which are stored in the first memory region of memory on the host computing system using copy-on-write operations.
5. The method of claim 1 , further comprising executing a page operations logger process on the host computing system, wherein the page operations logger is configured to generate the metadata that is stored and maintained in the metadata log residing in the second memory region of memory on the host computing system, and wherein the page operations logger is configured to transmit the checkpoint request message, which is issued by the process, to the data backup system.
6. The method of claim 1 , wherein the metadata pulled from the host computing system comprises a checkpoint record comprising information that specifies at least one of (i) a storage tier on the data backup system in which the checkpoint image should be stored and (ii) a data processing operation to be performed on the checkpoint image by the data backup system.
7. The method of claim 1 , further comprising:
receiving, by the data backup system, a page count threshold message transmitted from the host computing system, wherein the page count threshold message notifies the data backup system that a number of memory pages residing in the first memory region of memory on the host computing system, which have not yet been backed up by the data backup system, has met a predefined threshold;
in response to the page count threshold message, the data backup system performing a process comprising:
pulling a copy of metadata from the metadata log residing in the second memory region of memory on the host computing system, which is associated with the memory pages which have not yet been backed up by the data backup system;
pulling a copy of the memory pages from the first memory region of memory on the host computing system, which have not yet been backed up by the data backup system; and
generating, by the data backup system, a preliminary checkpoint image using the copies of the memory pages and the associated metadata pulled from the host computing system.
8. The method of claim 7 , wherein the page count threshold message notifies the data backup system that a number of modified memory pages residing in the first memory region of memory on the host computing system, which have not yet been backed up by the data backup system, has met a predefined threshold.
9. The method of claim 1 , wherein the first memory region and the second memory region comprise designated memory regions on a same memory device.
10. An article of manufacture comprising a processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code is executable by one or more processors to implement a process comprising:
receiving, by a data backup system, notification of a checkpoint request issued by a process executing on a host computing system to store a copy of a checkpoint image comprising one or more memory pages that reside in a first memory region of memory on the host computing system;
pulling, by the data backup system, a copy of metadata from a metadata log residing in a second memory region of memory on the host computing system, wherein the metadata is associated with the one or more memory pages of the checkpoint image;
pulling, by the data backup system, a copy of the one or more memory pages of the checkpoint image from the first memory region of memory on the host computing system;
generating, by the data backup system, a copy of the checkpoint image using the copies of the one or more memory pages and the associated metadata pulled from the host computing system; and
storing, by the data backup system, the copy of the checkpoint image in a storage device on the data backup system;
wherein the copy of the checkpoint image stored in the storage device on the data backup system comprises a same in-memory format as the checkpoint image residing in the first memory region of memory on the host computing system.
11. The article of manufacture of claim 10 , wherein the program code for pulling a copy of the metadata from the metadata log residing in the second memory region of memory on the host computing system comprises program code that is executable by the one or more processors for performing, by the data backup system, at least one direct memory access (RDMA) read operation to transfer the copy of the metadata from a non-volatile random access memory device on the host computing system to a non-volatile random access memory device on the data backup system; and
wherein the program code for pulling a copy of the one or more memory pages of the checkpoint image from the first memory region of memory on the host computing system comprises program code that is executable by the one or more processors for performing, by the data backup system, one or more RDMA read operations to transfer the copy of the memory pages from a RDMA-enabled non-volatile memory on the host computing system to the non-volatile random access memory device on the data backup system.
12. The article of manufacture of claim 10 , wherein the one or more memory pages of the checkpoint image comprise modified memory pages which are stored in the first memory region of memory on the host computing system using copy-on-write operations.
13. The article of manufacture of claim 10 , further comprising program code that is executable by the one or more processors for executing a page operations logger process on the host computing system, wherein the page operations logger is configured to generate the metadata that is stored and maintained in the metadata log residing in the second memory region of memory on the host computing system, and wherein the page operations logger is configured to transmit the checkpoint request message, which is issued by the process, to the data backup system.
14. The article of manufacture of claim 10 , wherein the metadata pulled from the host computing system comprises a checkpoint record comprising information that specifies at least one of (i) a storage tier on the data backup system in which the checkpoint image should be stored and (ii) a data processing operation to be performed on the checkpoint image by the data backup system.
15. The article of manufacture of claim 10 , further comprising program code that is executable by the one or more processors for performing a process comprising:
receiving, by the data backup system, a page count threshold message transmitted from the host computing system, wherein the page count threshold message notifies the data backup system that a number of memory pages residing in the first memory region of memory on the host computing system, which have not yet been backed up by the data backup system, has met a predefined threshold;
in response to the page count threshold message, the data backup system performing a process comprising:
pulling a copy of metadata from the metadata log residing in the second memory region of memory on the host computing system, which is associated with the memory pages which have not yet been backed up by the data backup system;
pulling a copy of the memory pages from the first memory region of memory on the host computing system, which have not yet been backed up by the data backup system; and
generating, by the data backup system, a preliminary checkpoint image using the copies of the memory pages and the associated metadata pulled from the host computing system.
16. The article of manufacture of claim 15 , wherein the page count threshold message notifies the data backup system that a number of modified memory pages residing in the first memory region of memory on the host computing system, which have not yet been backed up by the data backup system, has met a predefined threshold.
17. A system, comprising:
a storage device;
at least one processor; and
a memory to store program code that is executable by the at least one processor to implement a data backup system, wherein the data backup system is configured to:
receive notification of a checkpoint request issued by a process executing on a host computing system to store a copy of a checkpoint image comprising one or more memory pages that reside in a first memory region of memory on the host computing system;
pull a copy of metadata from a metadata log residing in a second memory region of memory on the host computing system, wherein the metadata is associated with the one or more memory pages of the checkpoint image;
pull a copy of the one or more memory pages of the checkpoint image from the first memory region of memory on the host computing system;
generate a copy of the checkpoint image using the copies of the one or more memory pages and the associated metadata pulled from the host computing system; and
store the copy of the checkpoint image in the storage device;
wherein the copy of the checkpoint image stored in the storage device comprises a same in-memory format as the checkpoint image residing in the first memory region of memory on the host computing system.
18. The system of claim 17 , wherein the data backup system is configured to:
pull a copy of the metadata from the metadata log residing in the second memory region of memory on the host computing system by performing at least one direct memory access (RDMA) read operation to transfer the copy of the metadata from a non-volatile random access memory device on the host computing system to a non-volatile random access memory device on the data backup system; and
pull a copy of the one or more memory pages of the checkpoint image from the first memory region of memory on the host computing system by performing one or more RDMA read operations to transfer the copy of the memory pages from a RDMA-enabled non-volatile memory on the host computing system to the non-volatile random access memory device on the data backup system.
19. The system of claim 17 , wherein the one or more memory pages of the checkpoint image comprises modified memory pages which are stored in the first memory region of memory on the host computing system using copy-on-write operations.
20. The system of claim 17 , wherein the data backup system is further configured to:
receive a page count threshold message transmitted from the host computing system, wherein the page count threshold message notifies the data backup system that a number of memory pages residing in the first memory region of memory on the host computing system, which have not yet been backed up by the data backup system, has met a predefined threshold; and
in response to the page count threshold message, the data backup system is configured to:
pull a copy of metadata from the metadata log residing in the second memory region of memory on the host computing system, which is associated with the memory pages which have not yet been backed up by the data backup system;
pull a copy of the memory pages from the first memory region of memory on the host computing system, which have not yet been backed up by the data backup system; and
generate a preliminary checkpoint image using the copies of the memory pages and the associated metadata pulled from the host computing system.