SYSTEMS AND METHODS FOR CONSISTENT BACKUP OF DISTRIBUTED, TRANSACTIONAL DATABASES
A distributed, transactional database uses timestamps, such as logical clock values, for entry versioning and transaction management in the database. To write to the database, a service requests a timestamp to be inserted into the database with a new version of data. During a backup procedure, a cleanup process is paused, issuing new timestamps is paused, and a backup timestamp is generated, which results in an effective backup copy. During a restore of a backup, a snapshot of the database is loaded and any entries older than the backup timestamp are deleted, which ensures that a consistent restore has occurred.
1 . A database system comprising:
one or more computer hardware processors configured to execute computer-executable instructions to cause the database system to at least:
identify a portion of a duplicate copy of data from a plurality of nodes of a database cluster;
determine whether the portion of the duplicate copy of data is changed;
in response to determining the portion of the duplicate copy is not changed, store metadata indicating that the portion of the duplicate copy of data is unchanged; and
in response to determining the portion of the duplicate copy of data is changed, store, in a backup data store, the portion of the duplicate copy and a backup timestamp.
2 . The database system of claim 1 , wherein the duplicate copy of data is generated in response to a backup request and during a pause on a cleanup process on the plurality of nodes of the database cluster.
3 . The database system of claim 1 , wherein the one or more computer hardware processors are further configured to execute the computer-executable instructions to cause the database system to at least:
pause a cleanup process on the plurality of nodes of the database cluster; and
during the pause of the cleanup process, generate the duplicate copy of data from the plurality of nodes.
4 . The database system of claim 1 , wherein determining whether the portion of the duplicate copy of data is changed comprises:
comparing the portion of the duplicate copy of data to a corresponding portion of a previous duplicate copy.
5 . The database system of claim 4 , wherein the comparing is based on hash values.
6 . The database system of claim 4 , wherein the portion of the duplicate copy of data comprises a first portion and/or a last portion of the duplicate copy of data.
7 . The database system of claim 1 , wherein determining whether the portion of the duplicate copy of data is changed comprises:
comparing a hash value of the portion of the duplicate copy of data to a hash value of a corresponding portion of a previous duplicate copy.
8 . The database system of claim 7 , wherein the portion of the duplicate copy of data comprises a first portion and/or a last portion of the duplicate copy of data, and wherein the portion of the corresponding portion of the previous duplicate copy comprises a first portion and/or a last portion of the previous duplicate copy of data.
9 . The database system of claim 1 , wherein the metadata indicates that a same portion of the duplicate copy of data is associated with multiple discrete backups.
10 . The database system of claim 1 , wherein the one or more computer hardware processors are further configured to execute the computer-executable instructions to cause the database system to at least:
in response to determining that one or more portions of backup data in the backup data store are not referenced by one or more items of metadata, delete the one or more portions of backup data.
11 . A method comprising:
identifying a portion of a duplicate copy of data from a plurality of nodes of a database cluster;
determining whether the portion of the duplicate copy of data is changed;
in response to determining the portion of the duplicate copy is not changed, storing metadata indicating that the portion of the duplicate copy of data is unchanged; and
in response to determining the portion of the duplicate copy of data is changed, storing, in a backup data store, the portion of the duplicate copy and a backup timestamp.
12 . The method of claim 11 , wherein the duplicate copy of data is generated in response to a backup request and during a pause on a cleanup process on the plurality of nodes of the database cluster.
13 . The method of claim 11 further comprising:
pause a cleanup process on the plurality of nodes of the database cluster; and
during the pause of the cleanup process, generate the duplicate copy of data from the plurality of nodes.
14 . The method of claim 11 , wherein determining whether the portion of the duplicate copy of data is changed comprises:
comparing the portion of the duplicate copy of data to a corresponding portion of a previous duplicate copy.
15 . The method of claim 14 , wherein the comparing is based on hash values.
16 . The method of claim 14 , wherein the portion of the duplicate copy of data comprises a first portion and/or a last portion of the duplicate copy of data.
17 . The method of claim 11 , wherein determining whether the portion of the duplicate copy of data is changed comprises:
comparing a hash value of the portion of the duplicate copy of data to a hash value of a corresponding portion of a previous duplicate copy.
18 . The method of claim 17 , wherein the portion of the duplicate copy of data comprises a first portion and/or a last portion of the duplicate copy of data, and wherein the portion of the corresponding portion of the previous duplicate copy comprises a first portion and/or a last portion of the previous duplicate copy of data.
19 . The method of claim 11 , wherein the metadata indicates that a same portion of the duplicate copy of data is associated with multiple discrete backups.
20 . The method of claim 11 further comprising:
in response to determining that one or more portions of backup data in the backup data store are not referenced by one or more items of metadata, deleting the one or more portions of backup data.