Network block device based continuous replication for Kubernetes container management systems
View Patent ↗Providing continuous replication for container management system that allows configuration of a volume as a replicated network block device (NBD) volume to an actual backend volume. The system configures a pod with an NBD container running and with the actual backend volume device attached. An NBD server intercepts all I/O data arriving to the NBD volume and a filter driver intercepts writes to the NBD volume and writes them to the actual backend volume. The intercepted I/O data is also sent to a remote replication server container with persistent and journal volumes. The data is applied to the replication volume using the journal as a recover point for recovery for any point in time.
1. A computer-implemented method of providing continuous replication for a container management system, comprising:
automatically configuring a pod having a network block device (NBD) container maintained by an NBD server;
accessing a back-end device attached to the NBD server as a persistent volume on a software-defined storage;
intercepting input/output (I/O) communication to the NBD server to expose a volume over the NBD;
intercepting, in a filter driver running within the NBD server, writes to the exposed volume and writing them to the persistent volume;
sending the intercepted I/O communication to a journal and a remote replication server container which has the persistent volume attached thereto;
identifying a pod to be replicated in a failover operation; and
applying data of the I/O communication to the replication volume server container using the journal to allow recovery for any point in time for the failover operation.
2. The method of claim 1 further comprising encapsulating a volume that the pod with the NBD container so that the pod can interact with an NBD volume of the NBD container.
3. The method of claim 2 further comprising performing the failover operation by:
shutting down all containers running in the pod;
selecting a point-in-time on the replication server;
causing the replication server to roll the data to the selected point in time;
configuring the NBD server and connecting it to a replica volume accessible from the replication server to expose the NBD volume; and
reconfiguring the pod to attach to a new NBD filter prior to restart of the pod.
4. The method of claim 3 further comprising adding, through an orchestration engine, a replication of the replication process seamlessly using the NBD filter.
5. The method of claim 1 wherein the software-defined storage comprises a DellEMC® ScaleIO™ software-defined storage system that creates a server-based storage area network (SAN) from local application server storage using certain servers or hardware.
6. The method of claim 1 wherein the NBD server includes a splitter component splitting the intercepted I/Os to the NBD container and to a remote replication server container.
7. The method of claim 1 wherein the journal comprises a data log stored on a storage medium accessible to the replication server.
8. The method of claim 1 wherein the container management system comprises a Kubernetes system, and wherein the persistent volume is implemented as a Kubernetes PersistentVolume (PV) in a cluster.
9. A computer-implemented method of providing continuous replication for container management system, comprising:
intercepting, in a network block device (NBD) server input/output (I/O) communications from a containerized application having an application volume;
sending the intercepted I/Os to an NBD container on the NBD server, a replica container, and a journal container on a replication server functionally coupled to the NBD server;
exposing an NBD volume on the NBD container to a persistent volume of the NBD server; and
applying, by the replication server, continuous replication using the replica container and the journal container to replicate the application volume.
10. The method of claim 9 wherein the container management system comprises a Kubernetes system, and wherein the replica container and journal container are implemented as Kubernetes PersistentVolumes (PVs).
11. The method of claim 10 wherein the application volume represents a container mapped volume in user space that is mapped upon the NBD client block device in kernel space.
12. The method of claim 11 further comprising:
identifying a to be replicated; and
encapsulating a volume that the pod is meant to write to with the NBD container so that the identified pod can interact with an NBD volume of the NBD container.
13. The system of claim 12 wherein the journal comprises a data log stored on a storage medium accessible to the replication server.
14. The method of claim 9 wherein the persistent volume of the NBD server is maintained in a software-defined storage system.
15. The method of claim 14 wherein the software-defined storage system comprises a DellEMC® ScaleIO™ software-defined storage system that creates a server-based storage area network (SAN) from local application server storage using certain servers or hardware.
16. A system comprising a processor-based executable module configured to provide continuous replication for container management system, comprising:
a network block device (NBD) of an NBD server having a filter driver intercepting input/output (I/O) communications from a containerized application having an application volume;
a splitter of the NBD server sending the intercepted I/Os to an NBD container on the NBD server, a replica container, and a journal container on a replication server functionally coupled to the NBD server, exposing an NBD volume on the NBD container to a persistent volume of the NBD server; and
a replication process executed on the replication server applying continuous replication using the replica container and the journal container to replicate the application volume.
17. The system of claim 16 wherein the container management system comprises a Kubernetes system, and wherein the replica container and journal container are implemented as Kubernetes PersistentVolumes (PVs).
18. The system of claim 17 further comprising an orchestration engine allowing a data protection service to add replication seamlessly using the NBD filter driver.
19. The system of claim 16 wherein the replication process further:
identifies a pod that is to be replicated is identified; and
encapsulates a volume that the pod writes to with the NBD container so that the identified pod can interact with an NBD volume of the NBD container.
20. The system of claim 19 wherein the persistent volume of the NBD server is maintained in a software-defined storage system, and wherein the software-defined storage system comprises a DellEMC® ScaleIO™ software-defined storage system that creates a server-based storage area network (SAN) from local application server storage using certain servers or hardware.