IP Library Granted Patent US 11,860,741
Granted Patent B2
US 11,860,741 · App. 17/687,431 · Granted Jan 2, 2024

Continuous data protection

Inventors: Akshat Vig (Seattle, WA); Tate Andrew Certain (Seattle, WA); Go Hori (Seattle, WA)
Assignee: Amazon Technologies, Inc.
G06F11/1451G06F9/54G06F11/1469G06F16/2358G06F16/278G06F2201/84
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,860,741
App. No.
17/687,431
Granted
Jan 2, 2024
Kind
B2
Abstract

Changes made to a database table are accumulated, in durable storage, and snapshots of partitions of the table are obtained. For successive snapshots of a partition, the system accesses a previous snapshot, applies changes from the accumulated changes, and stores the updated snapshot to a durable data store. The accumulated changes and the successive partition snapshots are made available to restore the database to any point in time across a continuum between successive snapshots. Although each partition of the table may have a backup snapshot that was generated at a time different from when other partition snapshots were generated, changes from respective change logs may be selectively log-applied to distinct partitions of a table to generate an on-demand backup of the entire table at common point-in-time across partitions. Point-in-time restores of a table may rely upon a similar process to coalesce partition snapshots that are not aligned in time.

Claims (78)

1. A system, comprising:

one or more processors and memory configured to implement a data manager for a database service that stores data in partitions, wherein change log data for the partitions is accumulated in a change log data store, and wherein for a split of a partition at a point-in-time into two or more split partitions, a corresponding change log for the partition is split into two or more split change logs, the data manager configured to, for generation of a backup for one or more of the split partitions:

split a previous snapshot, corresponding to the partition prior to the split, into one or more split snapshots corresponding to the one or more of the split partitions for which a backup is being generated; and

for one or more of the split snapshots:

apply some of the changes, from the corresponding change log that are prior to the point-in-time, and changes from a respective split change log, to the respective split snapshot to create an updated split snapshot; and

store, or provide, the respective updated split snapshot as the backup for the respective split partition.

2. The system as recited in claim 1 , wherein:

the one or more split snapshots comprise at least first and second split snapshots; and

to generate the backup for the at least first and second split snapshots the data manager is configured to:

split the corresponding change log for the partition prior to the split into at least first and second change logs, wherein the corresponding change log for the partition prior to the split is split at the point-in-time into the at least first and second change logs such that each of the at least first and second change logs include changes since the split for the respective partition;

for each of the at least first and second split snapshots:

perform said apply some of the changes; and

perform said store, or provide, the respective updated split snapshot as the backup for the respective split partition.

3. The system as recited in claim 1 , wherein:

the change logs and snapshots are stored to durable storage in a native format of the durable storage and converted to a non-native format when needed;

the data manager is configured to:

send, with each of the updated one or more split snapshots in non-native format, respective checksums for verifying whether the updated one or more split snapshots have been received without error; and

receive one or more response messages indicating unsuccessful storage of the updated one or more split snapshots.

4. The system as recited in claim 3 , wherein the data manager is configured to retry, responsive to the one or more messages indicating unsuccessful storage, said store the respective updated split snapshot.

5. The system as recited in claim 1 ,

wherein to store the respective updated split snapshot as the backup for the respective split partition the data manager is configured to store the respective updated split snapshot to a durable storage data store in a native format of the data store.

6. The system as recited in claim 1 , wherein:

the change logs and snapshots are stored to a durable storage in a native format of the durable storage;

to perform said apply some of the changes, from the corresponding change log that are prior to the point-in-time, and changes from the respective split change log, to the respective split snapshot to create the updated split snapshot, the data manager is configured to:

obtain, in a non-native format:

the changes from the corresponding change log that are prior to the point-in-time, and

the changes from the respective split change log; and

log-apply the obtained non-native format changes to the respective split snapshot to create the updated split snapshot.

7. The system as recited in claim 1 , wherein:

the data manager is configured to initiate said generation of the backup for one or more of the split partitions during performance of a partition split technique that splits the partition prior to the split into the one or more split partitions.

8. A method, comprising:

performing, by one or more computing devices for a database service that stores data in partitions, wherein change log data for the partitions is accumulated in a change log data store, and wherein for a split of a partition at a point-in-time into two or more split partitions, a corresponding change log for the partition is split into two or more split change logs:

generating a backup for one or more of the split partitions, comprising:

splitting a previous snapshot, corresponding to the partition prior to the split, into one or more split snapshots corresponding to the one or more of the split partitions for which a backup is being generated;

for one or more of the split snapshots:

applying some of the changes, from the corresponding change log that are prior to the point-in-time, and changes from a respective split change log, to the respective split snapshot to create an updated split snapshot; and

storing, or providing, the respective updated split snapshot as the backup for the respective split partition.

9. The method as recited in claim 8 , wherein:

the one or more split snapshots comprise at least first and second split snapshots; and

said generating the backup comprises:

splitting the corresponding change log for the partition prior to the split into at least first and second change logs, wherein the corresponding change log for the partition prior to the split is split at the point-in-time into the at least first and second change logs such that each of the at least first and second change logs include changes since the split for the respective split partitions;

for each of the at least first and second split snapshots:

performing said applying some of the changes; and

performing said storing, or providing, the respective updated split snapshot as the backup for the respective split partition.

10. The method as recited in claim 8 , wherein:

the change logs and snapshots are stored to durable storage in a native format of the durable storage and converted to a non-native format when needed;

the method further comprises:

sending, with each of the updated one or more split snapshots in the non-native format, respective checksums for verifying whether the updated one or more split snapshots have been received without error; and

receiving one or more messages indicating unsuccessful storage of the updated one or more split snapshots.

11. The method as recited in claim 10 , further comprising, responsive to the one or more messages indicating unsuccessful storage, retrying said store the respective updated split snapshot.

12. The method recited in claim 8 , wherein said storing the respective updated split snapshot as the backup for the respective split partition comprising storing the respective updated split snapshot to a durable storage data store in a native format of the durable storage data store.

13. The method recited in claim 8 , further comprising:

storing the change logs and snapshots to a durable storage in a native format of the durable storage;

converting, responsive to a request for one or more of the change logs or for one or more of the snapshots, the requested one or more of the change logs or one or more snapshots from the durable storage to a non-native format; and

responding to the request with the one or more of the change logs or the one or more snapshots in the non-native format.

14. The method recited in claim 8 , further comprising

initiating said generation of the backup for one or more of the split partitions during performance of a partition split technique that splits the partition prior to the split into the one or more split partitions.

15. One or more non-transitory computer readable storage media storing program instruction that, if executed on or across one or more computers, cause the one or more computers to implement a data manager configured to:

generate a backup for one or more split partitions of a database service that stores data in partitions, wherein change log data for the partitions is accumulated in a change log data store, wherein for a split of one of the partitions at a point-in-time into two or more split partitions, a corresponding change log for the partition is split into two or more split change logs, and wherein to generate the backup the data manager is configured to:

split a previous snapshot, corresponding to the partition prior to the split, into one or more split snapshots corresponding to the one or more of the split partitions for which a backup is being generated;

for one or more of the split snapshots:

apply some of the changes, from the corresponding change log that are prior to the point-in-time, and changes from a respective split change log, to the respective split snapshot to create an updated split snapshot; and

store, or provide, the respective updated split snapshot as the backup for the respective split partition.

16. The one or more non-transitory computer readable storage media as in claim 15 , wherein:

the one or more split snapshots comprise at least first and second split snapshots; and

to generate the backup for the at least first and second split snapshots, the program instructions, if executed, configure the data manager to:

split the corresponding change log for the partition prior to the split into at least first and second change logs, wherein the corresponding change log for the partition prior to the split is split at the point-in-time into the at least first and second change logs such that each of the at least first and second change logs include changes since the split for the respective partition;

for each of the at least first and second split snapshots:

perform said apply some of the changes; and

perform said store, or provide, the respective updated split snapshot as the backup for the respective split partition.

17. The one or more non-transitory computer readable storage media as in claim 15 , wherein:

the change logs and snapshots are stored to durable storage in a native format of the durable storage and converted to a non-native format when needed;

the program instructions, if executed, configure the data manager to:

send, with each of the updated one or more split snapshots in non-native format, respective checksums for verifying whether the updated one or more split snapshots have been received without error; and

receive one or more response messages indicating unsuccessful storage of the updated one or more split snapshots.

18. The one or more non-transitory computer readable storage media as in claim 17 , wherein the program instructions, if executed, configure the data manager to retry, responsive to the one or more messages indicating unsuccessful storage, said store the respective updated split snapshot.

19. The one or more non-transitory computer readable storage media as in claim 15 , wherein to store the respective updated split snapshot as the backup for the respective split partition, the program instructions, if executed, configure the data manager to store the respective updated split snapshot to a durable storage data store in a native format of the data store.

20. The one or more non-transitory computer readable storage media as in claim 15 , wherein the program instructions, if executed, configure the data manager to initiate said generation of the backup for one or more of the split partitions during performance of a partition split technique that splits the partition prior to the split into the one or more split partitions.

Continuity (3)
Continuation 15920207 · Mar 13, 2018
Provisional Application 62590223 · Nov 22, 2017
Related Publication 20220188196A1 · Jun 16, 2022