Systems and methods for consistent backup of distributed, transactional databases
A distributed, transactional database uses timestamps, such as logical clock values, for entry versioning and transaction management in the database. To write to the database, a service requests a timestamp to be inserted into the database with a new version of data. During a backup procedure, a cleanup process is paused, issuing new timestamps is paused, and a backup timestamp is generated, which results in an effective backup copy. During a restore of a backup, a snapshot of the database is loaded and any entries older than the backup timestamp are deleted, which ensures that a consistent restore has occurred.
1. A database system comprising:
a database cluster comprising a plurality of nodes, wherein each node of the plurality of nodes comprises a key-value table, and wherein each entry of a respective key-value table comprises a timestamp; and
one or more computer hardware processors configured to execute computer-executable instructions to cause the database system to at least:
receive a request to restore one or more key-value tables on the plurality of nodes;
retrieve, based on the request to restore and from a backup data store, a duplicate copy of data to be restored;
load the duplicate copy of data on a first node of the plurality of nodes; and
delete a first entry from the first node with a first timestamp more recent than a backup timestamp, wherein the backup timestamp was generated during a hold on the database system issuing new timestamps to one or more external computing devices during a backup process.
2. The database system of claim 1 , wherein the request to restore includes an indication of the first node.
3. The database system of claim 1 , wherein the request to restore includes an indication of the first timestamp.
4. The database system of claim 1 , wherein the backup timestamp is retrieved from the backup data store.
5. The database system of claim 1 , wherein loading the duplicate copy of data comprises:
changing entries of a first key-value table of the first node during a hold on writing of new entries to the first key-value table.
6. The database system of claim 1 , wherein the request to restore includes an indication of the first node and an indication of the first timestamp.
7. The database system of claim 6 , wherein the backup timestamp is retrieved from the backup data store.
8. The database system of claim 7 , wherein loading the duplicate copy of data comprises:
changing entries of a first key-value table of the first node during a hold on writing of new entries to the first key-value table.
9. A computer-implemented method comprising:
receiving a request to restore one or more key-value tables on a plurality of nodes of a database cluster, wherein each node of the plurality of nodes comprises a key-value table, and wherein each entry of a respective key-value table comprises a timestamp;
retrieving, based on the request to restore and from a backup data store, a duplicate copy of data to be restored;
loading the duplicate copy of data on a first node of the plurality of nodes; and
deleting a first entry from the first node with a first timestamp more recent than a backup timestamp, wherein the backup timestamp was generated during a hold on allowing writes from committing to the database cluster during a backup process,
wherein the computer-implemented method is performed by one or more computer hardware processors.
10. The computer-implemented method of claim 9 , wherein the request to restore includes an indication of the first node.
11. The computer-implemented method of claim 9 , wherein the request to restore includes an indication of the first timestamp.
12. The computer-implemented method of claim 9 , wherein the backup timestamp is retrieved from the backup data store.
13. The computer-implemented method of claim 9 , wherein loading the duplicate copy of data comprises:
changing entries of a first key-value table of the first node during a hold on writing of new entries to the first key-value table.
14. The computer-implemented method of claim 9 , wherein the request to restore includes an indication of the first node and an indication of the first timestamp.
15. The computer-implemented method of claim 14 , wherein the backup timestamp is retrieved from the backup data store.
16. The computer-implemented method of claim 15 , wherein loading the duplicate copy of data comprises:
changing entries of a first key-value table of the first node during a hold on writing of new entries to the first key-value table.