Providing consistent database recovery after database failure for distributed databases with non-durable storage leveraging background synchronization point

1. A method comprising:

generating a plurality of change records corresponding to a plurality of database transactions executed by a multi-node database system against a database, wherein each node of said multi-node database system generates a respective set of change records of said plurality of change records;

said multi-node database system generating a plurality of synchronization points that each correspond to a respective global timestamp of said multi-node database system, wherein for each synchronization point of said plurality of synchronization points, each node of said multi-node database system has persistently stored all change records of the respective set of change records of said each node for any database transaction of said plurality of database transactions that has committed before or no later than the respective global timestamp of said each synchronization point;

for a particular synchronization point that corresponds to a particular global timestamp of said multi-node database system, recovering the database by at least, for a particular node of said multi-node database system, applying at least a respective subset of the respective set of change records of said particular node to roll forward one or more database transactions committed before or no later than said particular global timestamp; and

wherein the method is performed by one or more computing devices.

2. The method of claim 1 , wherein generating a plurality of synchronization points includes, for said particular synchronization point, said multi-node database system ensuring change records for any database transactions being executed by said multi-node database system against the database that have committed at or before said particular synchronization point have been durably stored.

3. The method of claim 2 , wherein said database transactions being executed by said multi-node database system against the database that have committed at or before said particular synchronization point include a particular database transaction, wherein said ensuring change records have been durably stored includes flushing change records for said particular database transaction to a persistent log, where said particular database transaction was committed before commencing flushing.

4. The method of claim 1 , wherein the method further includes:

a particular node of said multi-node database system receiving a request to establish a synchronization point,

in response to said particular node receiving a request to establish a synchronization point, durably storing change records for one or more database transactions of said plurality of database transactions that have committed at or before said particular synchronization point.

5. The method of claim 4 , wherein before commencing durably storing change records said one or more database transactions have been committed.

6. The method of claim 1 , wherein the method further includes said multi-node database system non-durably committing said plurality of database transactions.

7. The method of claim 1 , wherein generating said particular synchronization point of said plurality of synchronization points comprises assigning a timestamp to the particular synchronization point.

8. The method of claim 7 , wherein assigning said timestamp to said particular synchronization point comprises:

determining a plurality of largest timestamps, wherein determining a plurality of largest timestamps includes, determining, for each node of said multi-node database system, a respective largest timestamp of a database transaction of said plurality of database transactions that has been committed by said each node;

determining a maximum timestamp among said plurality of largest timestamps; and

assigning a timestamp larger than said maximum timestamp to said particular synchronization point.

9. The method of claim 1 , wherein recovering said database includes:

determining that one or more particular change records are for a database transaction that is committed after said particular synchronization point; and

in response to determining that one or more particular change records are for a database transaction that is committed after said particular synchronization point, forgoing applying said particular change records to a checkpoint image.

10. The method of claim 1 , wherein said database comprises a plurality of database shards, each database shard of said plurality of database shards being managed by a respective node of said multi-node database system.

11. One or more non-transitory computer-readable media storing instructions which, when executed by one or more processors, cause:

generating a plurality of change records corresponding to a plurality of database transactions executed by a multi-node database system against a database, wherein each node of said multi-node database system generates a respective set of change records of said plurality of change records;

said multi-node database system generating a plurality of synchronization points that each corresponds to a respective global timestamp of said multi-node database system, wherein for each synchronization point of said plurality of synchronization points, each node of said multi-node database system has persistently stored all change records of the respective set of change records of said each node for any database transaction of said plurality of database transactions that has committed before or no later than the respective global timestamp of said each synchronization point;

for a particular synchronization point that corresponds to a particular global timestamp of said multi-node database system, recovering the database by at least, for a particular node of said multi-node database system, applying at least a respective subset of the respective set of change records of said particular node to roll forward one or more database transactions committed before or no later than said particular global timestamp.

12. The one or more non-transitory computer-readable media of claim 11 , further comprising instructions, which when executed by the one or more processors, cause:

ensuring change records for any database transactions being executed by said multi-node database system against the database that have committed at or before said particular synchronization point have been durably stored.

13. The one or more non-transitory computer-readable media of claim 12 , wherein said database transactions being executed by said multi-node database system against database that have committed at or before said particular synchronization point include a particular database transaction, wherein said ensuring change records have been durably stored includes flushing change records for said particular database transaction to a persistent log, where said particular database transaction was committed before commencing flushing.

14. The one or more non-transitory computer-readable media of claim 11 , further comprising instructions, which when executed by the one or more processors, cause:

receiving a request to establish a synchronization point a particular node of said multi-node database system,

in response to said particular node receiving a request to establish a synchronization point, durably storing change records for one or more database transactions of said plurality of database transactions that have committed at or before said particular synchronization point.

15. The one or more non-transitory computer-readable media of claim 14 , further comprising instructions which, when executed by the one or more processors, cause:

durably storing change records said one or more database transactions have been committed before commencing.

16. The one or more non-transitory computer-readable media of claim 11 , further comprising instructions, which when executed by the one or more processors, cause said multi-node database system non-durably committing said plurality of database transactions.

17. The one or more non-transitory computer-readable media of claim 11 , further comprising instructions, which when executed by the one or more processors, cause:

assigning a timestamp to the particular synchronization point.

18. The one or more non-transitory computer-readable media of claim 17 , further comprising instructions, which when executed by the one or more processors, cause:

determining a plurality of largest timestamps, wherein determining a plurality of largest timestamps includes, determining, for each node of said multi-node database system, a respective largest timestamp of a database transaction of said plurality of database transactions that has been committed by said each node;

determining a maximum timestamp among a plurality of largest timestamps; and

assigning a timestamp larger than said maximum timestamp to said particular synchronization point.

19. The one or more non-transitory computer-readable media of claim 11 , further comprising instructions, which when executed by the one or more processors, cause:

determining that one or more particular change records are for a database transaction that is committed after said particular synchronization point; and

in response to determining that one or more particular change records are for a database transaction that is committed after said particular synchronization point, forgoing applying said particular change records to a checkpoint image.

20. The one or more non-transitory computer-readable media of claim 11 , wherein said database comprises a plurality of database shards, each database shard of said plurality of database shards being managed by a respective node of said multi-node database system.