Continuous data protection using retroactive backup snapshots
Methods, storage systems and computer program products create a transactionally consistent backup of a database by generating, during a time period beginning with a first time and ending with a second time, a copy of the database including multiple tables. During the time period, queries that request updates to the tables are periodically generated and conveyed to a software system managing the database, responses including updates to the tables are received responsively to the periodic queries, and the received updates are stored to a journal. A set of the updates not stored in the copy are identified, in the journal, and copy of the database is updated with the updates so as to ensure that the copy includes the transactionally consistent backup of the database.
1 . A method for use with a database comprising multiple tables, comprising:
during a time period beginning with a first time and ending with a second time, copying the database to a file;
receiving, from a software system managing the database, updates made to the tables during the time period, and storing the received updates to a journal including a raw journal and a set of journal partitions into which the raw journal is split; and
by accessing a set of partition records, which specify a lowest key value and a highest key value in each of the journal partitions:
identifying, in the journal, a set of the updates not stored in the file; and
updating the file, by a processor, with the set of the updates so that the file comprises a transactionally consistent backup of the database.
2 . The method according to claim 1 , wherein the journal and the file comprise respective structured text files.
3 . The method according to claim 2 , wherein the structured text files comprise comma-separated value (CSV) files.
4 . The method according to claim 1 , wherein the journal and the file comprise respective binary files.
5 . The method according to claim 1 , wherein the journal comprises one or more JavaScript Object Notation (JSON) files.
6 . The method according to claim 1 , further comprising generating and conveying, to the software system, queries that request the updates, wherein receiving the updates comprises receiving the updates by receiving responses to the queries that include the updates.
7 . The method according to claim 6 , wherein generating and conveying the queries comprises generating and conveying the queries during the time period.
8 . The method according to claim 6 , wherein generating and conveying the queries comprises periodically generating and conveying the queries.
9 . The method according to claim 6 , wherein the software system comprises a customer relationship management (CRM) application, and wherein the queries comprise calls to an application programming interface (API) of the CRM application.
10 . The method according to claim 9 , wherein the tables comprise respective sets of fields, and wherein a given response comprises a plurality of updates to one or more of the fields in one or more of the tables.
11 . An apparatus for use with a database comprising multiple tables, comprising:
a memory, configured to store;
a journal including a raw journal and a set of journal partitions into which the raw journal is split, and
a set of partition records, which specify a lowest key value and a highest key value in each of the journal partitions; and
one or more processors configured:
to copy the database, during a time period beginning with a first time and ending with a second time, to a file,
to receive, from a software system managing the database, updates made to the tables during the time period, and to store the received updates to the journal, and
by accessing the set of partition records:
to identify, in the journal, a set of the updates not stored in the file, and
to update the file with the set of the updates so that the file comprises a transactionally consistent backup of the database.
12 . A computer software product comprising a tangible non-transitory computer-readable medium in which program instructions are stored, which instructions, when read by one or more processors, cause the processors:
to copy to a file, during a time period beginning with a first time and ending with a second time, a database comprising multiple tables,
to receive, from a software system managing the database, updates made to the tables during the time period, and to store the received updates to a journal including a raw journal and a set of journal partitions into which the raw journal is split, and
by accessing a set of partition records, which specify a lowest key value and a highest key value in each of journal partitions:
to identify, in the journal, a set of the updates not stored in the file, and
to update the file with the set of the updates so that the file comprises a transactionally consistent backup of the database.
13 . The computer software product according to claim 12 , wherein the journal and the file comprise respective structured text files.
14 . The computer software product according to claim 13 , wherein the structured text files comprise comma-separated value (CSV) files.
15 . The computer software product according to claim 12 , wherein the journal and the file comprise respective binary files.
16 . The computer software product according to claim 12 , wherein the journal comprises one or more JavaScript Object Notation (JSON) files.
17 . The computer software product according to claim 12 , wherein the processors are further configured to generate and convey, to the software system, queries that request the updates, wherein the processors are configured to receive the updates by receiving responses to the queries that include the updates.
18 . The computer software product according to claim 17 , wherein the processors are configured to generate and convey the queries during the time period.
19 . The computer software product according to claim 17 , wherein the software system comprises a customer relationship management (CRM) application, and wherein the queries comprise calls to an application programming interface (API) of the CRM application.
20 . The computer software product according to claim 17 , wherein the tables comprise respective sets of fields, and wherein a given one of the responses includes updates to one or more of the fields in one or more of the tables.