Multi-channel backup system
Methods, systems, and computer programs are presented for backing up data to a distributed multichannel storage system. One method includes an operation for detecting, by a distributed backup system (DBS), a request for backing up data from a host to the distributed backup system. The DBS sends a request to a backup agent executing at the host to initiate the backup. The backup agent mounts an endpoint for writing to the DBS and executes a script that writes the data for the backup to the endpoint. Further, the method includes operations for creating a file system stack at the DBS associated with the endpoint, storing data received by the file system stack at the DBS, and detecting, by the DBS, an end of the backup. The DBS sends a command to the backup agent to end the backup and terminates the file system stack at the DBS.
1. A computer-implemented method comprising:
detecting, by a distributed backup system, a request for backing up data from a host to the distributed backup system;
sending a request, from the distributed backup system to a backup agent executing at the host, to initiate a backup, wherein, based on the request to initiate the backup, the backup agent:
mounts a first network file share endpoint at the host for writing to the distributed backup system and executes a first script that writes a first portion of the data for the backup to the first network file share endpoint; and
mounts a second network file share endpoint at the host for writing to the distributed backup system and executes a second script that writes a second portion of the data for backup to the second network file share endpoint;
creating, based at least in part on sending the request to initiate the backup:
a first file system stack at a first node of the distributed backup system that provides a first path to the first network file share endpoint at the host via a first network file share client at the distributed backup system, wherein the first network file share client is at the first node; and
a second file system stack at a second node of the distributed backup system that provides a second path to the second network file share endpoint at the host via a second network file share client at the distributed backup system, wherein the second network file share client is at the second node,
storing data received by the first file system stack and the second file system stack at the distributed backup system, wherein the first portion of the data for the backup is received by the first file system stack at the first node, and wherein the second portion of the data for the backup is received by the second file system stack at the second node;
detecting, by the distributed backup system, an end of the backup;
sending a command, in response to detecting the end of the backup and from the distributed backup system to the backup agent executing at the host, to end the backup; and
terminating the first file system stack and the second file system stack at the distributed backup system in response to detecting the end of the backup.
2. The computer-implemented method as recited in claim 1 , further comprising:
in response to the command to end the backup, unmounting the first network file share endpoint by the backup agent.
3. The computer-implemented method as recited in claim 1 , wherein the distributed backup system comprises a plurality of nodes, wherein the first file system stack is created at one node from the plurality of nodes, wherein an internet protocol (IP) address of the one node with the first file system stack is provided to the backup agent.
4. The computer-implemented method as recited in claim 3 , further comprising:
providing a plurality of channels for backing up data from the host to the plurality of nodes.
5. The computer-implemented method as recited in claim 1 , further comprising:
providing a plurality of endpoints at the distributed backup system, the plurality of endpoints being in communication with the first file system stack for backing up data.
6. The computer-implemented method as recited in claim 5 , wherein the plurality of endpoints comprises the first network file share endpoint and a Server Message Block (SMB) endpoint.
7. The computer-implemented method as recited in claim 1 , wherein the request for backing up data is generated by the distributed backup system based on a service level agreement (SLA).
8. The computer-implemented method as recited in claim 7 , wherein the SLA defines one or more of frequency of full backups, frequency of incremental backups, and amount of time to retain the full backups and the incremental backups.
9. The computer-implemented method as recited in claim 1 , wherein an availability of a managed volume for backing up the data comprises a plurality of states comprising export requested, exported, resize requested, unexport requested, and destroyed.
10. The computer-implemented method as recited in claim 1 , further comprising:
initializing parameters of the backup agent when backing up of data is initially configured by the distributed backup system, the parameters comprising an IP address of a node for backing up the data, type of endpoint, and the first script to be executed for backing up data.
11. The computer-implemented method as recited in claim 1 , further comprising:
creating, prior to creation of the first file system stack and based at least in part on sending the request to initiate the backup, a third file system stack at a third node of the distributed backup system that provides a third path to a third network file share endpoint at the host via a third network file share client at the third node, wherein the first file system stack is created at the first node of the distributed backup system based at least in part on a failure condition or a traffic condition at the third node.
12. The computer-implemented method as recited in claim 1 , wherein detecting the request for backing up data comprises determining, in accordance with a service level agreement, that a scheduled time for an ingest of data from the host is satisfied, wherein sending the request to initiate the backup is in response to determining that the scheduled time for the ingest of data from the host is satisfied.
13. A system comprising:
a memory comprising instructions; and
one or more computer processors, wherein the instructions, when executed by the one or more computer processors, cause the system to perform operations comprising:
detecting, by a distributed backup system, a request for backing up data from a host to the distributed backup system;
sending a request, from the distributed backup system to a backup agent executing at the host, to initiate a backup, wherein, based on the request to initiate the backup, the backup agent:
mounts a first network file share endpoint at the host for writing to the distributed backup system and executes a first script that writes a first portion of the data for the backup to the first network file share endpoint; and
mounts a second network file share endpoint at the host for writing to the distributed backup system and executes a second script that writes a second portion of the data for backup to the second network file share endpoint;
creating, based at least in part on sending the request to initiate the backup:
a first file system stack at a first node of the distributed backup system that provides a first path to the first network file share endpoint at the host via a first network file share client at the distributed backup system, wherein the first network file share client is at the first node; and
a second file system stack at a second node of the distributed backup system that provides a second path to the second network file share endpoint at the host via a second network file share client at the distributed backup system, wherein the second network file share client is at the second node,
storing data received by the first file system stack and the second file system stack at the distributed backup system, wherein the first portion of the data for the backup is received by the first file system stack at the first node, and wherein the second portion of the data for the backup is received by the second file system stack at the second node;
detecting, by the distributed backup system, an end of the backup;
sending a command, in response to detecting the end of the backup and from the distributed backup system to the backup agent executing at the host, to end the backup; and
terminating the first file system stack and the second file system stack at the distributed backup system in response to detecting the end of the backup.
14. The system as recited in claim 13 , wherein the instructions further cause the one or more computer processors to perform operations comprising:
in response to the command to end the backup, unmounting the first network file share endpoint by the backup agent.
15. The system as recited in claim 13 , wherein the distributed backup system comprises a plurality of nodes, wherein the first file system stack is created at one node from the plurality of nodes, wherein an internet protocol (IP) address of the one node with the first file system stack is provided to the backup agent.
16. The system as recited in claim 15 , wherein the instructions further cause the one or more computer processors to perform operations comprising:
providing a plurality of channels for backing up data from the host to the plurality of nodes.
17. The system as recited in claim 13 , wherein the instructions further cause the one or more computer processors to perform operations comprising:
providing a plurality of endpoints at the distributed backup system, the plurality of endpoints being in communication with the first file system stack for backing up data.
18. The system as recited in claim 13 , wherein detecting the request for backing up data comprises determining, in accordance with a service level agreement, that a scheduled time for an ingest of data from the host is satisfied, wherein sending the request to initiate the backup is in response to determining that the scheduled time for the ingest of data from the host is satisfied.
19. A non-transitory machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations comprising:
detecting, by a distributed backup system, a request for backing up data from a host to the distributed backup system;
sending a request, from the distributed backup system to a backup agent executing at the host, to initiate a backup, wherein, based on the request to initiate the backup, the backup agent:
mounts a first network file share endpoint at the host for writing to the distributed backup system and executes a first script that writes a first portion of the data for the backup to the first network file share endpoint; and
mounts a second network file share endpoint at the host for writing to the distributed backup system and executes a second script that writes a second portion of the data for backup to the second network file share endpoint;
creating, based at least in part on sending the request to initiate the backup:
a first file system stack at a first node of the distributed backup system that provides a first path to the first network file share endpoint at the host via a first network file share client at the distributed backup system, wherein the first network file share client is at the first node; and
a second file system stack at a second node of the distributed backup system that provides a second path to the second network file share endpoint at the host via a second network file share client at the distributed backup system, wherein the second network file share client is at the second node,
storing data received by the first file system stack and the second file system stack at the distributed backup system, wherein the first portion of the data for the backup is received by the first file system stack at the first node, and wherein the second portion of the data for the backup is received by the second file system stack at the second node;
detecting, by the distributed backup system, an end of the backup;
sending a command, in response to detecting the end of the backup and from the distributed backup system to the backup agent executing at the host, to end the backup; and
terminating the first file system stack and the second file system stack at the distributed backup system in response to detecting the end of the backup.
20. The non-transitory machine-readable storage medium as recited in claim 19 , wherein the instructions, when executed by the machine, cause the machine to further perform operations comprising:
in response to the command to end the backup, unmounting the first network file share endpoint by the backup agent.