Database table restoration for data with skewed distribution
A database management system obtains key ranges associated with partitions of a database table, and determines that restoration of data associated with at least one of these partitions would take time that is estimated to exceed a goal time. The database management system splits a partition into two or more additional partitions, so that the respective estimated restoration times for data associated with the additional partitions is estimated to be less than the goal time. The database management system then causes the database table to be restored according to partitions that comprises the additional partitions.
1. A system, comprising:
at least one processor; and
a memory comprising instructions that, in response to execution by the at least one processor, cause the system to at least:
obtain, from a backup of a database table, information indicative of key ranges associated with a first plurality of partitions of the database table hosted on a plurality of corresponding computing nodes;
determine a size of data associated with a partition of the first plurality of partitions of the database table;
estimate a duration required to restore data associated with the partition;
split, in response to a determination that the duration to restore the data associated with the partition is estimated to exceed a goal duration, the partition into two or more additional partitions until an estimated restore duration of the two or more additional partitions is less than the goal duration, wherein each of the two or more additional partitions are hosted on additional corresponding computing nodes, wherein the partition is split by at least mapping the key ranges associated with the first plurality of partitions to the two or more additional partitions; and
cause the database table to be restored according to a second plurality of partitions of the database table, wherein the second plurality of partitions comprises the two or more additional partitions.
2. The system of claim 1 , the memory comprising further instructions that, in response to execution by the at least one processor, cause the system to at least:
estimate the restore duration based at least in part on a write capacity of a storage device to store the data associated with the partition, and based at least in part on a size of the data associated with the partition.
3. The system of claim 1 , wherein the backup of the database table comprises one or more snapshots storing the data associated with the partition.
4. The system of claim 1 , the memory comprising further instructions that, in response to execution by the at least one processor, cause the system to at least:
obtain information indicative of a point-in-time for restoring the database table;
obtain the information indicative of key ranges associated with the first plurality of partitions, such that the information corresponds to key ranges in effect as of the point-in-time; and
determine the size of data associated with the partition as of the point-in-time.
5. The system of claim 1 , the memory comprising further instructions that, in response to execution by the at least one processor, cause the system to at least:
determine that a total number of partitions would exceed a maximum number of partitions; and
combine two or more additional partitions into one combined partition, wherein the second plurality of partitions comprises the combined partition.
6. A computer-implemented method, comprising:
obtaining information indicative of key ranges associated with a first plurality of partitions of a database table hosted on two or more corresponding nodes, the database table to be restored according to a second plurality of partitions;
determining that restoration of data associated with a partition of the first plurality of partitions of the database table is estimated to take an amount of time that exceeds a goal duration;
in response to the determination, splitting the partition into two or more additional partitions until respective estimated restoration duration for data associated with the two or more additional partitions is estimated to be less than the goal duration, wherein each of the two or more additional partitions are hosted on additional corresponding nodes, wherein splitting the partition comprises mapping the key ranges associated with the first plurality of partitions to key ranges associated with the two or more additional partitions; and
causing the database table to be restored according to the second plurality of partitions of the database table, wherein the second plurality of partitions comprises the two or more additional partitions.
7. The computer-implemented method of claim 6 , wherein the information indicative of key ranges associated with the first plurality of partitions is obtained from a backup of the database table.
8. The computer-implemented method of claim 6 , further comprising:
estimating an amount of time to restore the data associated with the partition based at least in part on write capacity of a storage device that is selected to store the data associated with the partition.
9. The computer-implemented method of claim 6 , further comprising:
estimating an amount of time to restore the data associated with the partition based at least in part on a size of the data associated with the partition.
10. The computer-implemented method of claim 6 , further comprising:
obtaining information indicative of size of the data associated with the partition from one or more files of a backup of the database table.
11. The computer-implemented method of claim 6 , further comprising:
obtaining information indicative of a size of the data associated with the partition from a storage device on which the data associated with the partition is stored.
12. The method of claim 6 , wherein the information indicative of key ranges associated with the first plurality of partitions corresponds to key ranges in effect as of a point-in-time associated with a requested restoration of the table.
13. The method of claim 6 , further comprising: estimating an amount of time to restore the data associated with the partition based at least in part on data stored in the database table at a point-in-time associated with a requested restoration of the table.
14. A non-transitory computer-readable storage medium storing thereon executable instructions that, as a result of being executed by one or more processors of a computer system, cause the computer system to at least:
determine key ranges associated with a first plurality of partitions of a database table hosted on a plurality of corresponding nodes, the database table to be restored according to a second plurality of partitions;
determine that restoration of data associated with a partition of the first plurality of partitions of the database table is estimated to take an amount of time exceeding a goal amount of time;
in response to the determination, divide the partition into two or more additional partitions until respective estimated restoration times for respective data associated with the two or more additional partitions is estimated to be less than the goal amount of time, each of the two or more additional partitions being hosted on additional corresponding nodes, wherein the divided partition comprises an updated mapping of the key ranges to the two or more additional partitions; and
initiate restoration of the database table according to the second plurality of partitions, wherein the second plurality of partitions comprises the two or more additional partitions.
15. The non-transitory computer-readable storage medium of claim 14 , the non-transitory computer-readable storage medium storing thereon further executable instructions that, as a result of being execute by the one or more processors, cause the computer system to at least:
obtain information indicative of the key ranges associated with the first plurality of partitions from a backup of the database table.
16. The non-transitory computer-readable storage medium of claim 14 , the non-transitory computer-readable storage medium storing thereon further executable instructions that, as a result of being executed by the one or more processors, cause the computer system to at least:
estimate an amount of time to restore the partition based at least in part on write capacity of a storage device to store data associated with the partition upon restoration, and based at least in part on a size of the data associated with the partition.
17. The non-transitory computer-readable storage medium of claim 14 , wherein a size of the data associated with the partition is determined based, at least in part, on one or more files of a backup of the partition.
18. The non-transitory computer-readable storage medium of claim 14 , wherein the key ranges associated with the first plurality of partitions corresponds to key ranges in effect as of a point-in-time associated with a requested restoration of the table.
19. The non-transitory computer-readable storage medium of claim 14 , the non-transitory computer-readable storage medium storing thereon further executable instructions that, as a result of being executed by the one or more processors, cause the computer system to at least:
estimate an amount of time to restore the data associated with the partition based at least in part on data stored in the table at a point-in-time associated with a requested restoration of the table.
20. The non-transitory computer-readable storage medium of claim 14 , wherein the partition is recursively divided into additional partitions until respective sizes of data associated with the additional partitions are each indicative of a restoration time estimated to be less than the goal amount of time.