Datacenter workflow automation scenarios using virtual databases
View Patent ↗Information from multiple databases is retrieved and stored on a database storage system. Multiple point-in-time copies are obtained for each database. A point-in-time copy retrieves data changed in the database since the retrieval of a previous point-in-time copy. A virtual database (VDB) is created by creating a set of files in the data storage system. Each file in the set of files created for a VDB is linked to the database blocks on the database storage system associated with a point-in-time copy of the source database. The set of files associated with the VDB are mounted on a database server allowing the database server to read from and write to the set of files. Workflows based on VDBs allow various usage scenarios based on databases to be implemented efficiently, for example, testing and development, backup and recovery, and data warehouse building.
1. A method for replicating a database, the method comprising:
receiving, by a first storage system, database blocks for different point-in-time copies of a source database;
storing, by the first storage system, point-in-time copies of the source database, wherein a point-in-time copy shares one or more stored database blocks with one or more other point-in time-copies;
creating, by the first storage system, a first virtual database comprising database blocks stored on the first storage system, the virtual database sharing one or more database blocks with other virtual databases of the first storage system;
providing, by the first storage system, access to the first virtual database to a first database server;
replicating, by the first storage system, the database blocks from the first storage system to a second storage system;
creating, by the second storage system, a second virtual database comprising database blocks stored on the second storage system, the second virtual database matching data of the first virtual database, the second virtual database sharing one or more database blocks with other virtual databases of the second storage system;
receiving, by the first storage system, modifications to the first virtual database from the first database server;
sending, by the first storage system, information describing changes to the first virtual database to the second storage system; and
refreshing, by the second storage system, the second virtual database, such that the data of the second virtual matches the data of the first virtual database.
2. The method of claim 1 , wherein replicating database blocks from the first storage system to the second storage system comprises incrementally copying updates in the first storage system to the second storage system.
3. The method of claim 1 , wherein creating the first virtual database comprises:
creating a first set of files, each file in the first set of files linked to the database blocks on the first storage system associated with a point-in-time copy of the source database; and
mounting the first set of files associated with the first virtual database on the first database server.
4. The method of claim 1 , further comprising:
providing access to the second virtual database to a second database server;
receiving modifications to the second virtual database from the second database server; and
sending information describing changes to the second virtual database to the first storage system.
5. The method of claim 1 , wherein creating the second virtual database comprises:
creating a second set of files, each file in the second set of files linked to the database blocks on the second storage system associated with a point-in-time copy of the source database; and
mounting the second set of files associated with the second virtual database on a second database server.
6. The method of claim 1 , further comprising:
copying program code to the second storage system, the program code comprising one or more software modules configured to create virtual databases.
7. The method of claim 1 , wherein refreshing the second virtual database comprises:
shutting down the second virtual database;
updating data of the second virtual database based on the updates of the first virtual database; and
restarting the virtual database.
8. The method of claim 1 , further comprising:
receiving, by the first storage system, information describing transaction logs from the source database wherein the transaction logs represent changes to the source database since a previous point-in-time copy was received; and
wherein creating the first virtual database comprises applying the transaction logs to database blocks of a point-in-time copy associated with the first virtual database.
9. The method of claim 1 , wherein receiving point-in-time copies comprises receiving data streams corresponding to the point-in-time copies, wherein each data stream comprises data from database blocks associated with the source database.
10. The method of claim 9 , further comprising:
analyzing the data streams received to identify database blocks; and
storing the identified database blocks.
11. The method of claim 9 , further comprising:
analyzing the data streams received to identify database blocks and analyzing metadata of database blocks to determine a length of the database blocks for processing the data stream.
12. The method of claim 11 , further comprising:
analyzing metadata of database blocks to determine whether to store the identified database block.
13. The method of claim 12 , further comprising:
determining not to store the database blocks that are identified as one of temporary database blocks, empty database blocks, or database blocks that did not change since a previous retrieval of point-in-time copy of the source database.
14. A computer readable non-transitory storage medium, storing instructions for:
receiving, by a first storage system, database blocks for different point-in-time copies of a source database;
storing, by the first storage system, point-in-time copies of the source database, wherein a point-in-time copy shares one or more stored database blocks with one or more other point-in time-copies;
creating, by the first storage system, a first virtual database comprising database blocks stored on the first storage system, the virtual database sharing one or more database blocks with other virtual databases of the first storage system;
providing, by the first storage system, access to the first virtual database to a first database server;
replicating, by the first storage system, the database blocks from the first storage system to a second storage system;
creating, by the second storage system, a second virtual database comprising database blocks stored on the second storage system, the second virtual database matching data of the first virtual database, the second virtual database sharing one or more database blocks with other virtual databases of the second storage system;
receiving, by the first storage system, modifications to the first virtual database from the first database server;
sending, by the first storage system, information describing changes to the first virtual database to the second storage system; and
refreshing, by the second storage system, the second virtual database, such that the data of the second virtual matches the data of the first virtual database.
15. The computer readable non-transitory storage medium of claim 14 , wherein replicating database blocks from the first storage system to the second storage system comprises incrementally copying updates in the first storage system to the second storage system.
16. The computer readable non-transitory storage medium of claim 14 , wherein creating the first virtual database comprises:
creating a first set of files, each file in the first set of files linked to the database blocks on the first storage system associated with a point-in-time copy of the source database; and
mounting the first set of files associated with the first virtual database on the first database server.
17. The computer readable non-transitory storage medium of claim 14 , further comprising:
providing access to the second virtual database to a second database server;
receiving modifications to the second virtual database from the second database server; and
sending information describing changes to the second virtual database to the first storage system.
18. The computer readable non-transitory storage medium of claim 14 , further comprising:
copying program code to the second storage system, the program code comprising one or more software modules configured to create virtual databases.
19. The computer readable non-transitory storage medium of claim 14 , wherein refreshing the second virtual database comprises:
shutting down the second virtual database;
updating data of the second virtual database based on the updates of the first virtual database; and
restarting the virtual database.
20. A computer-implemented system comprising:
a computer processor; and
a computer-readable non-transitory storage medium storing instructions for:
receiving, by a first storage system, database blocks for different point-in-time copies of a source database;
storing, by the first storage system, point-in-time copies of the source database, wherein a point-in-time copy shares one or more stored database blocks with one or more other point-in time-copies;
creating, by the first storage system, a first virtual database comprising database blocks stored on the first storage system, the virtual database sharing one or more database blocks with other virtual databases of the first storage system;
providing, by the first storage system, access to the first virtual database to a first database server;
replicating, by the first storage system, the database blocks from the first storage system to a second storage system;
creating, by the second storage system, a second virtual database comprising database blocks stored on the second storage system, the second virtual database matching data of the first virtual database, the second virtual database sharing one or more database blocks with other virtual databases of the second storage system;
receiving, by the first storage system, modifications to the first virtual database from the first database server;
sending, by the first storage system, information describing changes to the first virtual database to the second storage system; and
refreshing, by the second storage system, the second virtual database, such that the data of the second virtual matches the data of the first virtual database.