Integration of database with distributed storage system
Methods, systems, and computer programs are presented for providing backup services to a database. One method includes operations for installing a backup agent in a first database, receiving information about the first database, and executing, by the backup agent, queries to the first database to determine a topology of the first database. Further, the method includes configuring, based on the topology, a receiver service of a second database for backing up the first database in the second database. The backup agent configures an interface module of the first database to back up the first database to the second database. The configuration includes an interface to the receiver service of the second database and connection information for storing data in one or more nodes of the second database. The interface module streams updates from the first database to the second database based on the configuration of the interface module.
1. A computer-implemented method comprising:
installing a backup agent in a first database system;
receiving information about the first database system;
executing, by the backup agent; queries to the first database system to determine a topology of the first database system;
configuring, based on the topology, a receiver service of a second database system for backing up the first database system in the second database system,
configuring, by the backup agent, an interface module of the first database system to back up the first database system to the second database system, the configuration including an interface to the receiver service of the second database system and connection information for storing data in one or more nodes of the second database system; and
streaming; from the interface module; updates from the first database system to the second database system based on the configuration of the interface module.
2. The method as recited in claim 1 , wherein the second database system provides full backups for a given timestamp and incremental backups based on the full backups.
3. The method as recited in claim 1 , wherein the connection information includes a list of internet protocol (IP) address of the nodes of the second database system, wherein the list of IP addresses is updated over time by the backup agent based on a state of the second database system.
4. The method as recited in claim 3 , wherein streaming updates from the first database system comprises selecting randomly one from the list of IP addresses for storing the updates.
5. The method as recited in claim 1 , wherein updates from the first database system comprise transaction logs.
6. The method as recited in claim 1 , wherein streaming updates from the first database system comprises:
sending a prepare file backup command to the receiver service that returns a file handle;
writing data to the receiver service utilizing the file handle; and
sending a finish file backup to the receiver service.
7. The method as recited in claim 6 , wherein the prepare file backup command includes an identifier of a file object and metadata associated with the file object.
8. The method as recited in claim 6 , wherein writing data to the receiver service includes sending the file handle, an offset within a file object, and the data written to the file object.
9. The method as recited in claim 1 , wherein the information about the first database system comprises hostnames of the first database system, identifier of the first database system, and user credentials for accessing the first database system.
10. The method as recited in claim 1 , wherein the queries to the first database system include SQL queries comprising: query for active status of one or more databases; query for IP addresses of one or more hosts; query for determining a log mode; and query for size of a backup.
11. A system comprising:
a memory comprising instructions; and
one or more computer processors, wherein the instructions, when executed by the one or more computer processors, cause system perform operations comprising:
installing a backup agent in a first database system,
receiving information about the first database system;
executing, by the backup agent, queries to the first database system to determine a topology of the first database system;
configuring, based on the topology, a receiver service of a second database system for backing up the first database system in the second database system;
configuring, by the backup agent, an interface module of the first database system to back up the first database system to the second database system, the configuration including an interface to the receiver service of the second database system and connection information for storing data in one or more nodes of the second database system; and
streaming, from the interface module, updates from the first database system to the second database system based on the configuration of the interface module.
12. The system as recited in claim 11 , wherein the second database system provides full backups for a given timestamp and incremental backups based on the full backups.
13. The system as recited in claim 11 , wherein the connection information includes a list of internet protocol (IP) address of the nodes of the second database system, wherein the list of IP addresses is updated over time by the backup agent based on a state of the second database system.
14. The system as recited in claim 13 , wherein streaming updates from the first database system comprises selecting randomly one from the list of IP addresses for storing the updates.
15. The system as recited in claim 11 , wherein the updates from the first database system comprise transaction logs.
16. A tangible machine-readable storage medium including instructions that, when executed by a machine, cause the machine to perform operations comprising:
installing a backup agent in a first database system;
receiving information about the first database system;
executing, by the backup agent, queries to the first database system to determine a topology of the first database system;
configuring, based on the topology, a receiver service of a second database system for backing up the first database system in the second database system;
configuring, by the backup agent, an interface module of the first database system to back up the first database system to the second database system, the configuration including an interface to the receiver service of the second database system and connection information for storing data in one or more nodes of the second database system; and
streaming, from the interface module, updates from the first database system to the second database system based on the configuration of the interface module.
17. The tangible machine-readable storage medium as recited in claim 16 , wherein the second database system provides full backups for a given timestamp and incremental backups based on the full backups.
18. The tangible machine-readable storage medium as recited in claim 16 , wherein the connection information includes a list of internet protocol (IP) address of the nodes of the second database system, wherein the list of IP addresses is updated over time by the backup agent based on a state of the second database system.
19. The tangible machine-readable storage medium as recited in claim 18 , wherein streaming updates from the first database system comprises selecting randomly one from the list of IP addresses for storing the updates.
20. The tangible machine-readable storage medium as recited in claim 16 , wherein the updates from the first database system comprise transaction logs.