Datacenter workflow automation scenarios using virtual databases
View Patent ↗Information from multiple databases is retrieved and stored on a database storage system. Multiple point-in-time copies are obtained for each database. A point-in-time copy retrieves data changed in the database since the retrieval of a previous point-in-time copy. A virtual database (VDB) is created by creating a set of files in the data storage system. Each file in the set of files created for a VDB is linked to the database blocks on the database storage system associated with a point-in-time copy of the source database. The set of files associated with the VDB are mounted on a database server allowing the database server to read from and write to the set of files. Workflows based on VDBs allow various usage scenarios based on databases to be implemented efficiently, for example, testing and development, backup and recovery, and data warehouse building.
1. A method for refreshing a virtual database, the method comprising:
receiving, by a storage system, database blocks for different point-in-time copies of a source database, wherein each database block includes metadata associated with the database block;
transforming, by the storage system, data of a received database block to a format distinct from a format of the received database block;
storing, by a storage system, point-in-time copies of the source database, wherein a point-in-time copy shares one or more stored database blocks with one or more other point-in time-copies, the point-in-time copy comprising database blocks storing the transformed data;
creating a virtual database comprising database blocks based on a first point-in-time copy of the source database stored on the storage system, the virtual database sharing one or more database blocks with other virtual databases of the storage system;
generating one or more reports based on the data of the virtual database;
receiving a request to refresh the virtual database, the request to refresh received after generating the one or more reports;
responsive to receiving the request to refresh the virtual database, refreshing the virtual database comprising:
identifying a latest point-in-time copy representing a recent point-in-time copy of the source database stored in the storage system, and
updating the virtual database to use database blocks of the latest point-in-time copy; and
responsive to refreshing the virtual database, refreshing the one or more reports, the refreshing comprising regenerating the one or more reports of the virtual database based on the latest point-in-time copy.
2. The method of claim 1 , wherein refreshing the virtual database further comprises:
applying transaction logs to the database blocks of the latest point-in-time copy, wherein the transaction logs were received after the storage system received the latest point-in-time copy.
3. The method of claim 1 , wherein refreshing the virtual database further comprises:
shutting down the virtual database before updating the virtual database to use database blocks of the latest point-in-time copy; and
restarting the virtual database after updating the virtual database to use database blocks of the latest point-in-time copy.
4. The method of claim 1 , wherein the method further comprises:
receiving a schedule for updating the virtual database; and
refreshing the virtual database based on the schedule.
5. The method of claim 4 , wherein the schedule specifies refreshing the virtual database periodically.
6. The method of claim 1 , wherein the method further comprises:
receiving information identifying a postscript operation; and
performing the postscript operation after refreshing the virtual database.
7. The method of claim 1 , wherein creating the virtual database comprises:
creating a set of files linked to the stored database blocks of the first point-in-time copy, and
mounting the set of files to a target system allowing a database server running on the target system to access the set of files.
8. The method of claim 1 , wherein transforming the data of a received database block comprises compressing data stored in the received database block.
9. The method of claim 1 , wherein the storage system analyzes each database block to determine whether the database block is stored in the storage system.
10. A non-transitory computer readable storage medium storing instructions for:
receiving, by a storage system, database blocks for different point-in-time copies of a source database, wherein each database block includes metadata associated with the database block;
transforming, by the storage system, data of a received database block to a format distinct from a format of the received database block;
storing, by a storage system, point-in-time copies of the source database, wherein a point-in-time copy shares one or more stored database blocks with one or more other point-in-time copies, the point-in-time copy comprising database blocks storing the transformed data;
creating a virtual database comprising database blocks based on a first point-in-time copy of the source database stored on the storage system, the virtual database sharing one or more database blocks with other virtual databases of the storage system;
generating one or more reports based on the data of the virtual database;
receiving a request to refresh the virtual database, the request to refresh received after generating the one or more reports;
responsive to receiving the request to refresh the virtual database, refreshing the virtual database comprising:
identifying a latest point-in-time copy representing a recent point-in-time copy of the source database stored in the storage system, and
updating the virtual database to use database blocks of the latest point-in-time copy; and
responsive to refreshing the virtual database, refreshing the one or more reports, the refreshing comprising regenerating the one or more reports of the virtual database based on the latest point-in-time copy.
11. The non-transitory computer readable storage medium of claim 10 , wherein refreshing the virtual database further comprises:
applying transaction logs to the database blocks of the latest point-in-time copy, wherein the transaction logs were received after the storage system received the latest point-in-time copy.
12. The non-transitory computer readable storage medium of claim 10 , wherein refreshing the virtual database further comprises:
shutting down virtual database before updating the virtual database to use database blocks of the latest point-in-time copy; and
restarting the virtual database after updating the virtual database to use database blocks of the latest point-in-time copy.
13. The non-transitory computer readable storage medium of claim 10 , wherein the instructions are for further performing the steps of:
receiving a schedule for updating the virtual database; and
refreshing the virtual database based on the schedule.
14. The non-transitory computer readable storage medium of claim 13 , wherein the schedule specifies refreshing the virtual database periodically.
15. The non-transitory computer readable storage medium of claim 10 , wherein the instructions are for further performing the steps of:
receiving information identifying a postscript operation; and
performing the postscript operation after performing the refresh of the virtual database.
16. The non-transitory computer readable storage medium of claim 10 , wherein creating the virtual database comprises:
creating a set of files linked to the stored database blocks of the first point-in-time copy, and
mounting the set of files to a target system allowing a database server running on the target system to access the set of files.
17. The non-transitory computer readable storage medium of claim 10 , wherein the metadata associated with a database block comprises one or more of:
the size of the database block; or
a log sequence number based on an order in which database blocks are updated in the source database.
18. The non-transitory computer readable storage medium of claim 10 , wherein transforming the data of a received database block comprises compressing data stored in the received database block.
19. The non-transitory computer readable storage medium of claim 10 , wherein the storage system analyzes each database block to determine whether the database block is stored in the storage system.
20. A computer system comprising:
a processor; and
a non-transitory computer readable storage medium storing instructions for execution by the processor, the instructions for:
receiving, by a storage system, database blocks for different point-in-time copies of a source database, wherein each database block includes metadata associated with the database block;
transforming, by the storage system, data of a received database block to a format distinct from a format of the received database block;
storing, by a storage system, point-in-time copies of the source database, wherein a point-in-time copy shares one or more stored database blocks with one or more other point-in time-copies, the point-in-time copy comprising database blocks storing the transformed data;
creating a virtual database comprising database blocks based on a first point-in-time copy of the source database stored on the storage system, the virtual database sharing one or more database blocks with other virtual databases of the storage system;
generating one or more reports based on the data of the virtual database;
receiving a request to refresh the virtual database, the request to refresh received after generating the one or more reports;
responsive to receiving the request to refresh the virtual database, refreshing the virtual database comprising:
identifying a latest point-in-time copy representing a recent point-in-time copy of the source database stored in the storage system, and
updating the virtual database to use database blocks of the latest point-in-time copy; and
responsive to refreshing the virtual database, refreshing the one or more reports, the refreshing comprising regenerating the one or more reports of the virtual database based on the latest point-in-time copy.
21. The computer system of claim 20 , wherein refreshing the virtual database further comprises:
shutting down virtual database before updating the virtual database to use database blocks of the latest point-in-time copy; and
restarting the virtual database after updating the virtual database to use database blocks of the latest point-in-time copy.
22. The computer system of claim 20 , wherein the instructions are for further performing the steps of:
receiving a schedule for updating the virtual database; and
refreshing the virtual database based on the schedule.
23. The computer system of claim 22 , wherein the schedule specifies refreshing the virtual database periodically.
24. The computer system of claim 20 , wherein the instructions are for further performing the steps of:
receiving information identifying a postscript operation; and
performing the postscript operation after performing the refresh of the virtual database.
25. The computer system of claim 20 , wherein creating the virtual database comprises:
creating a set of files linked to the stored database blocks of the first point-in-time copy, and
mounting the set of files to a target system allowing a database server running on the target system to access the set of files.
26. The method of claim 1 , wherein the metadata associated with a database block comprises one or more of:
the size of the database block; or
a log sequence number based on an order in which database blocks are updated in the source database.
27. The computer system of claim 20 , wherein refreshing the virtual database further comprises:
applying transaction logs to the database blocks of the latest point-in-time copy, wherein the transaction logs were received after the storage system received the latest point-in-time copy.
28. The computer system of claim 20 , wherein the metadata associated with a database block comprises one or more of:
the size of the database block; or
a log sequence number based on an order in which database blocks are updated in the source database.
29. The computer system of claim 20 , wherein transforming the data of a received database block comprises compressing data stored in the received database block.
30. The computer system of claim 20 , wherein the storage system analyzes each database block to determine whether the database block is stored in the storage system.
31. A method for refreshing a virtual database, the method comprising:
receiving, by a storage system, database blocks for different point-in-time copies of a source database, wherein each database block includes metadata associated with the database block, wherein the source database is associated with an application, the application associated with an application specific data;
storing, by a storage system, point in time copies of the source database, wherein a point-in-time copy shares one or more stored database blocks with one or more other point-in time-copies;
creating a virtual database comprising database blocks based on a first point-in-time copy of the source database stored on the storage system, the virtual database sharing one or more database blocks with other virtual databases of the storage system, wherein the virtual database is provisioned to a target system executing an instance of the application;
receiving information identifying a prescript operation, the prescript operation configured to load the application specific data associated with the application for copying to the target system;
receiving a request to refresh the virtual database;
performing the prescript operation, comprising, loading the application specific data associated with the application;
responsive to receiving the request to refresh the virtual database, refreshing the virtual database comprising:
identifying a latest point-in-time copy representing a recent point-in-time copy of the source database stored in the storage system, and
updating the virtual database to use database blocks of the latest point-in-time copy; and
copying the application specific data associated with the application to the target system.
32. The method of claim 31 , wherein refreshing the virtual database further comprises:
applying transaction logs to the database blocks of the latest point-in-time copy, wherein the transaction logs were received after the storage system received the latest point-in-time copy.
33. The method of claim 31 , wherein refreshing the virtual database further comprises:
shutting down virtual database before updating the virtual database to use database blocks of the latest point-in-time copy; and
restarting the virtual database after updating the virtual database to use database blocks of the latest point-in-time copy.
34. The method of claim 31 , wherein the method further comprises:
receiving a schedule for updating the virtual database; and
refreshing the virtual database based on the schedule.
35. The method of claim 31 , wherein the method further comprises:
receiving information identifying a postscript operation; and
performing the postscript operation after performing the refresh of the virtual database.
36. The method of claim 31 , wherein the metadata associated with a database block comprises one or more of:
the size of the database block; or
a log sequence number based on an order in which database blocks are updated in the source database.
37. The method of claim 31 , wherein the storage system transforms data of a received database block to a format distinct from a format of the received database block and stores the transformed data.
38. The method of claim 37 , wherein transforming the data of a received database block comprises compressing data stored in the received database block.
39. The method of claim 31 , wherein the storage system analyzes each database block to determine whether the database block is stored in the storage system.
40. A non-transitory computer readable storage medium storing instructions for:
receiving, by a storage system, database blocks for different point-in-time copies of a source database, wherein each database block includes metadata associated with the database block, wherein the source database is associated with an application, the application associated with an application specific data;
storing, by a storage system, point in time copies of the source database, wherein a point-in-time copy shares one or more stored database blocks with one or more other point-in time-copies;
creating a virtual database comprising database blocks based on a first point-in-time copy of the source database stored on the storage system, the virtual database sharing one or more database blocks with other virtual databases of the storage system, wherein the virtual database is provisioned to a target system executing an instance of the application;
receiving information identifying a prescript operation, the prescript operation configured to load the application specific data associated with the application for copying to the target system;
receiving a request to refresh the virtual database;
performing the prescript operation, comprising, loading the application specific data associated with the application;
responsive to receiving the request to refresh the virtual database, refreshing the virtual database comprising:
identifying a latest point-in-time copy representing a recent point-in-time copy of the source database stored in the storage system, and
updating the virtual database to use database blocks of the latest point-in-time copy; and
copying the application specific data associated with the application to the target system.
41. The non-transitory computer readable storage medium of claim 40 , wherein refreshing the virtual database further comprises:
applying transaction logs to the database blocks of the latest point-in-time copy, wherein the transaction logs were received after the storage system received the latest point-in-time copy.
42. The non-transitory computer readable storage medium of claim 40 , wherein refreshing the virtual database further comprises:
shutting down virtual database before updating the virtual database to use database blocks of the latest point-in-time copy; and
restarting the virtual database after updating the virtual database to use database blocks of the latest point-in-time copy.
43. The non-transitory computer readable storage medium of claim 40 , further storing instructions for:
receiving a schedule for updating the virtual database; and
refreshing the virtual database based on the schedule.
44. The non-transitory computer readable storage medium of claim 40 , further storing instructions for:
receiving information identifying a postscript operation; and
performing the postscript operation after performing the refresh of the virtual database.
45. The non-transitory computer readable storage medium of claim 40 , wherein the metadata associated with a database block comprises one or more of:
the size of the database block; or
a log sequence number based on an order in which database blocks are updated in the source database.
46. The non-transitory computer readable storage medium of claim 40 , wherein the instructions cause the storage system to transform data of a received database block to a format distinct from a format of the received database block and stores the transformed data.
47. The non-transitory computer readable storage medium of claim 46 , wherein transforming the data of a received database block comprises compressing data stored in the received database block.
48. The non-transitory computer readable storage medium of claim 40 , wherein the instructions cause the storage system to analyze each database block to determine whether the database block is stored in the storage system.
49. A computer system comprising:
a computer processors, and
a non-transitory computer readable storage medium storing instructions for execution by the processor, the instructions for:
receiving, by a storage system, database blocks for different point-in-time copies of a source database, wherein each database block includes metadata associated with the database block, wherein the source database is associated with an application, the application associated with an application specific data;
storing, by a storage system, point in time copies of the source database, wherein a point-in-time copy shares one or more stored database blocks with one or more other point-in time-copies;
creating a virtual database comprising database blocks based on a first point-in-time copy of the source database stored on the storage system, the virtual database sharing one or more database blocks with other virtual databases of the storage system, wherein the virtual database is provisioned to a target system executing an instance of the application;
receiving information identifying a prescript operation, the prescript operation configured to load the application specific data associated with the application for copying to the target system;
receiving a request to refresh the virtual database;
performing the prescript operation, comprising, loading the application specific data associated with the application;
responsive to receiving the request to refresh the virtual database, refreshing the virtual database comprising:
identifying a latest point-in-time copy representing a recent point-in-time copy of the source database stored in the storage system, and
updating the virtual database to use database blocks of the latest point-in-time copy; and
copying the application specific data associated with the application to the target system.
50. The computer system of claim 49 , wherein refreshing the virtual database further comprises:
applying transaction logs to the database blocks of the latest point-in-time copy, wherein the transaction logs were received after the storage system received the latest point-in-time copy.
51. The computer system of claim 49 , wherein refreshing the virtual database further comprises:
shutting down virtual database before updating the virtual database to use database blocks of the latest point-in-time copy; and
restarting the virtual database after updating the virtual database to use database blocks of the latest point-in-time copy.
52. The computer system of claim 49 , wherein the non-transitory computer readable storage medium further stores instructions for:
receiving information identifying a postscript operation; and
performing the postscript operation after performing the refresh of the virtual database.
53. The computer system of claim 49 , wherein the metadata associated with a database block comprises one or more of:
the size of the database block; or
a log sequence number based on an order in which database blocks are updated in the source database.
54. The computer system of claim 49 , further storing instructions for:
receiving a schedule for updating the virtual database; and
refreshing the virtual database based on the schedule.
55. The computer system of claim 49 , wherein the instructions cause the storage system to transform data of a received database block to a format distinct from a format of the received database block and stores the transformed data.
56. The computer system of claim 55 , wherein transforming the data of a received database block comprises compressing data stored in the received database block.
57. The computer system of claim 49 , wherein the instructions cause the storage system to analyze each database block to determine whether the database block is stored in the storage system.