Virtual partitions in virtual databases
View Patent ↗Information from source databases is retrieved and stored on a database storage system. Multiple point-in-time copies are obtained for each database. A source database may contain partitions comprising sets of database tables. A partition is imported into a virtual database created. The database storage system validates the partition before importing, by checking whether the partition includes database tables that refer to database tables outside the partition. A partition imported in a virtual database may be rewinded to an older state or refreshed to a more recent state. Multiple partitions may be included in a virtual database and the state of each partition modified independent of the state of the remaining virtual database including the other partitions. Multiple versions of the same partition corresponding to different points in time are incorporated in the same virtual database, thereby allowing a query to process different versions of the same database table.
1. A method comprising:
receiving multiple snapshots of a source database, the source database comprising a plurality of partitions, the plurality of partitions comprising, a first partition comprising a first set of database tables and a second partition comprising a second set of database tables, wherein the second set of database tables is distinct from the first set of database tables;
storing on a storage system, database blocks for a plurality of different snapshots of the source database, wherein at least some of the stored database blocks are shared between multiple snapshots of the source database;
creating a virtual database based on the source database, the virtual database represented by a set of files, each file in the set of files linked to database blocks of a first snapshot of the source database, the virtual database comprising a plurality of virtual partitions, the plurality of virtual partitions comprising:
a first virtual partition corresponding to the first partition of the source database, wherein the first virtual partition shares database blocks with first virtual partitions of other virtual databases created based on the source database and
a second virtual partition corresponding to the second partition of the source database, wherein the second virtual partition shares database blocks with second virtual partitions of other virtual databases created based on the source database; and
modifying the virtual database such that the data stored in each virtual partition from the plurality of virtual partition corresponds to data of the source database at a distinct point in time, the modifying comprising:
modifying the first virtual partition of the virtual database such that the data of the first virtual partition corresponds to database blocks of a second snapshot storing data of the source database at a second point in time; and
modifying the second virtual partition of the virtual database such that the data of the second virtual partition corresponds to database blocks of a third snapshot storing data of the source database at a third point in time.
2. The method of claim 1 , wherein the set of files is a first set of files, further comprising:
creating a staging virtual database represented by a second set of files, each file in the second set of files linked to database blocks of the second snapshot of the source database;
wherein modifying the first virtual partition of the virtual database comprises modifying the set of files of the first virtual partition to use the data of the second set of files; and
validating the first virtual partition using the staging virtual database, the validating comprising, verifying if any database table of the virtual partition includes one or more foreign keys referring to database tables outside the staging database.
3. The method of claim 2 , further comprising:
determining that the first virtual partition of the staging virtual database is invalid if the virtual partition includes a database table having a foreign key to a database table outside the virtual partition; and
responsive to determining that the first virtual partition is invalid, sending information indicating failure to modify the data of the first virtual partition to correspond to database blocks of a second snapshot of the source database.
4. The method of claim 2 , further comprising:
determining that the virtual partition of the staging virtual database is valid if all foreign keys of database tables of the virtual partition refer to database tables within the virtual partition; and
responsive to determining that the virtual partition is valid, proceeding with processing the request by modifying the first virtual database.
5. The method of claim 2 , further comprising:
updating metadata representing the database tables of the virtual partition in the first virtual database based on metadata representing the virtual partition in the staging virtual database.
6. The method of claim 1 , further comprising:
receiving a request to modify the data of the second virtual partition to correspond to database blocks of a fourth snapshot of the source database; and
modifying the virtual database such that the data of the second virtual partition corresponds to the fourth snapshot, the modifying comprising:
identifying one or more files of the set of files of the virtual database storing data of the second virtual partition; and
modifying the one or more files to point to database blocks of the fourth snapshot.
7. The method of claim 1 , further comprising:
receiving a first schedule for the first virtual partition and a second schedule for the second virtual partition, each schedule for modifying the virtual partitions, wherein the first schedule is distinct from the second schedule; and
modifying the first virtual partition in accordance with the first schedule and modifying the second virtual partition in accordance with the second schedule.
8. The method of claim 7 , wherein the first schedule specifies a periodicity with which the database tables of the first virtual partition are refreshed and the second schedule specifies a periodicity with which the database tables of the second virtual partition are refreshed.
9. A method comprising:
receiving different point-in-time copies of a source database, the source database comprising a partition comprising a set of database tables;
storing on a storage system, database blocks for a plurality of different point-in-time copies of the source database, wherein at least some of the stored database blocks are shared between multiple point-in-time copies of the source database;
creating a virtual database represented by a set of files, each file in the set of files linked to database blocks of a first snapshot of the source database, the first snapshot corresponding to data of the source database at a first point in time, the virtual database comprising a virtual partition corresponding to the partition of the source database, the virtual partition sharing database blocks with at least another virtual partition of another virtual database;
receiving a request to modify the data of the virtual partition to correspond to database blocks of a second snapshot of the source database the second snapshot corresponding to data of the source database at a second point in time; and
modifying the virtual database such that the data of the database blocks of the virtual partition corresponds to data of the source database at the second point in time and the data of the remaining database blocks of the virtual database corresponds to data of the source database at the first point in time, the modifying comprising:
identifying one or more files of the set of files of the virtual database storing data of the database tables of the virtual partition; and
modifying the one or more files of the set of files to point to database blocks of the second snapshot.
10. The method of claim 9 , wherein the virtual database is a first virtual database and the set of files is a first set of files, further comprising:
creating a staging virtual database represented by a second set of files, each file in the second set of files linked to database blocks of the second snapshot of the source database;
wherein modifying the first virtual database comprises modifying the one or more files of the first set of files using the files from the second set of files; and
validating the virtual partition using the staging virtual database, the validating comprising, verifying if any database table of the virtual partition includes one or more foreign keys referring to database tables outside the staging database.
11. The method of claim 10 , further comprising:
determining that the virtual partition of the staging virtual database is invalid if the virtual partition of the staging virtual database is determined to include a database table having a foreign key to a database table outside the virtual partition; and
responsive to determining that the virtual partition of the staging virtual database is invalid, sending information indicating failure to modify the data of the virtual partition to correspond to database blocks of a second snapshot of the source database.
12. The method of claim 10 , further comprising:
determining that the virtual partition of the staging virtual database is valid if all foreign keys of database tables of the virtual partition refer to database tables within the virtual partition; and
responsive to determining that the virtual partition is valid, proceeding with processing the request by modifying the first virtual database.
13. The method of claim 10 , further comprising:
updating metadata representing the database tables of the virtual partition in the first virtual database based on metadata representing the virtual partition in the staging virtual database.
14. The method of claim 10 , wherein the virtual partition is a first partition and wherein the source database comprises a second partition comprising a second set of database tables, the method further comprising:
receiving a request to modify the data of the second partition to correspond to database blocks of a third snapshot of the source database; and
modifying the virtual database such that the data of the second partition corresponds to the third snapshot, the modifying comprising:
identifying one or more files of the second set of files of the virtual database storing data of the database tables of the virtual partition; and
modifying the one or more files of the second set of files to point to database blocks of the third snapshot.
15. The method of claim 9 , further comprising:
receiving a first schedule for modifying the virtual partition and a second schedule for modifying the remaining databases tables of the virtual database, wherein the first schedule is distinct from the second schedule; and
modifying the virtual partition in accordance with the first schedule and modifying the remaining database tables of the virtual database in accordance with the second schedule.
16. The method of claim 15 , wherein the first schedule specifies a periodicity with which the database tables of the virtual partition are refreshed.
17. A non-transitory computer readable storage medium storing instructions for:
receiving multiple snapshots of a source database, the source database comprising a plurality of partitions, the plurality of partitions comprising, a first partition comprising a first set of database tables and a second partition comprising a second set of database tables, wherein the second set of database tables is distinct from the first set of database tables;
storing on a storage system, database blocks for a plurality of different snapshots of the source database, wherein at least some of the stored database blocks are shared between multiple snapshots of the source database;
creating a virtual database based on the source database, the virtual database represented by a set of files, each file in the set of files linked to database blocks of a first snapshot of the source database, the virtual database comprising a plurality of virtual partitions, the plurality of virtual partitions comprising:
a first virtual partition corresponding to the first partition of the source database, wherein the first virtual partition shares database blocks with first virtual partitions of other virtual databases created based on the source database and
a second virtual partition corresponding to the second partition of the source database, wherein the second virtual partition shares database blocks with second virtual partitions of other virtual databases created based on the source database; and
modifying the virtual database such that the data stored in each virtual partition from the plurality of virtual partition corresponds to data of the source database at a distinct point in time, the modifying comprising:
modifying the first virtual partition of the virtual database such that the data of the first virtual partition corresponds to database blocks of a second snapshot storing data of the source database at a second point in time; and
modifying the second virtual partition of the virtual database such that the data of the second virtual partition corresponds to database blocks of a third snapshot storing data of the source database at a third point in time.
18. The non-transitory computer readable storage medium of claim 17 , wherein the set of files is a first set of files, wherein the stored instructions are further for:
creating a staging virtual database represented by a second set of files, each file in the second set of files linked to database blocks of the second snapshot of the source database,
wherein modifying the first virtual partition of the virtual database comprises modifying the set of files of the first virtual partition to use the data of the second set of files; and
validating the first virtual partition using the staging virtual database, the validating comprising, verifying if any database table of the virtual partition includes one or more foreign keys referring to database tables outside the staging database.
19. The non-transitory computer readable storage medium of claim 18 , wherein the stored instructions are further for:
determining that the first virtual partition of the staging virtual database is invalid if the virtual partition includes a database table having a foreign key to a database table outside the virtual partition; and
responsive to determining that the first virtual partition is invalid, sending information indicating failure to modify the data of the first virtual partition to correspond to database blocks of a second snapshot of the source database.
20. The non-transitory computer readable storage medium of claim 18 , wherein the stored instructions are further for:
determining that the virtual partition of the staging virtual database is valid if all foreign keys of database tables of the virtual partition refer to database tables within the virtual partition; and
responsive to determining that the virtual partition is valid, proceeding with processing the request by modifying the first virtual database.
21. The non-transitory computer readable storage medium of claim 17 , wherein the stored instructions are further for:
receiving a first schedule for the first virtual partition and a second schedule for the second virtual partition, each schedule for modifying the virtual partitions, wherein the first schedule is distinct from the second schedule; and
modifying the first virtual partition in accordance with the first schedule and modifying the second virtual partition in accordance with the second schedule.
22. A non-transitory computer readable storage medium storing instructions for:
receiving different point-in-time copies of a source database, the source database comprising a partition comprising a set of database tables;
storing on a storage system, database blocks for a plurality of different point-in-time copies of the source database, wherein at least some of the stored database blocks are shared between multiple point-in-time copies of the source database;
creating a virtual database represented by a set of files, each file in the set of files linked to database blocks of a first snapshot of the source database, the first snapshot corresponding to data of the source database at a first point in time, the virtual database comprising a virtual partition corresponding to the partition of the source database, the virtual partition sharing database blocks with at least another virtual partition of another virtual database;
receiving a request to modify the data of the virtual partition to correspond to database blocks of a second snapshot of the source database the second snapshot corresponding to data of the source database at a second point in time; and
modifying the virtual database such that the data of the database blocks of the virtual partition corresponds to data of the source database at the second point in time and the data of the remaining database blocks of the virtual database corresponds to data of the source database at the first point in time, the modifying comprising:
identifying one or more files of the set of files of the virtual database storing data of the database tables of the virtual partition; and
modifying the one or more files of the set of files to point to database blocks of the second snapshot.
23. The non-transitory computer readable storage medium of claim 22 , wherein the virtual database is a first virtual database and the set of files is a first set of files, wherein the stored instructions are further for:
creating a staging virtual database represented by a second set of files, each file in the second set of files linked to database blocks of the second snapshot of the source database,
wherein modifying the first virtual database comprises modifying the one or more files of the first set of files using the files from the second set of files; and
validating the virtual partition using the staging virtual database, the validating comprising, verifying if any database table of the virtual partition includes one or more foreign keys referring to database tables outside the staging database.
24. The non-transitory computer readable storage medium of claim 23 , wherein the stored instructions are further for:
determining that the virtual partition of the staging virtual database is invalid if the virtual partition of the staging virtual database is determined to include a database table having a foreign key to a database table outside the virtual partition; and
responsive to determining that the virtual partition of the staging virtual database is invalid, sending information indicating failure to modify the data of the virtual partition to correspond to database blocks of a second snapshot of the source database.
25. The non-transitory computer readable storage medium of claim 23 , wherein the stored instructions are further for:
determining that the virtual partition of the staging virtual database is valid if all foreign keys of database tables of the virtual partition refer to database tables within the virtual partition; and
responsive to determining that the virtual partition is valid, proceeding with processing the request by modifying the first virtual database.
26. The non-transitory computer readable storage medium of claim 22 , wherein the stored instructions are further for:
receiving a first schedule for modifying the virtual partition and a second schedule for modifying the remaining databases tables of the virtual database; and
modifying the virtual partition in accordance with the first schedule and modifying the remaining database tables of the virtual database in accordance with the second schedule.