Asynchronous replication of in-scope table data
A computer-implemented method includes writing a plurality of change records to a first set of redo records in a first transaction log file. The computer-implemented method further includes determining, while writing a change record to a redo record, that the change record includes changes made to data associated with an in-scope table. The computer-implemented method further includes updating the first transaction log file with information indicating that the redo record includes changes made to data associated with the in-scope table. The computer-implemented method further includes accessing the first transaction log file from storage in response to receiving a request for replicating changes made to data associated with in-scope tables. The computer-implemented method further includes scraping a first redo record in the first transaction log file in response to determining that the first redo record includes at least one change record associated with the in-scope table.
1. A computer-implemented method for asynchronous replication between a source database and a target database, the computer-implemented method comprising:
determining, by one or more processors, while writing a first change record to a first redo record of a first transaction log file, that the first change record includes at least one change made to a first database table of a source database that is in-scope for replication between the source database and a target database;
modifying, by one or more processors, a header of the first transaction log file to include information indicating that the first redo record includes a change made to a database table that is in-scope for replication between the source database and the target database; and
responsive to receiving, by one or more processors, an instruction to replicate changes made to database tables that are in-scope for replication between the source database and the target database:
determining, by one or more processors, to scrape the first redo record to extract the change made to the first database table of the source database from the first redo record based, at least in part, on accessing the information included in the header of the first transaction log file indicating that the first redo record includes a change made to a database table that is in-scope for replication between the source database and the target database.
2. The computer-implemented method of claim 1 , further comprising:
retrieving, by one or more processors, each change record in the plurality of change records from a memory buffer in sequential time order from when each change record was written to the memory buffer.
3. The computer-implemented method of claim 1 , further comprising:
scraping, by one or more processors, the first redo record for the change made to the database table of the source database; and
transmitting, by one or more processors, the change extracted from the first redo record to the target database for population with the target database.
4. The computer-implemented method of claim 1 , further comprising:
generating, by one or more processors, a flag in a header of the first redo record indicating that the first redo record includes a change made to a database table that is in-scope for replication between the source database and the target database.
5. The computer-implemented method of claim 4 , wherein determining, by one or more processors, to scrape the first redo record for the change made to the first database table of the source database is further based, at least in part, on:
identifying, by one or more processors, the flag in the header of the first redo record indicating that the first redo record includes a change made to a database table that is in-scope for replication between the source database and the target database.
6. The computer-implemented method of claim 1 , further comprising:
determining, by one or more processors, while writing a second change record to a second redo record of the first transaction log file that the second change record only includes changes made to a second database table of the source target database that is out-of-scope for replication between the source database and the target database; and
modifying, by one or more processors, the header of the first transaction log file to include information indicating that the second redo record only includes changes made to a database table that is out-of-scope for replication between the source database and the target database.
7. The computer-implemented method of claim 6 , further comprising:
generating, by one or more processors, a flag in a header of the second redo record indicating that the second redo record only includes changes made to a database table that is out-of-scope for replication between the source database and the target database.
8. The computer-implemented method of claim 6 , further comprising:
determining, by one or more processors, to skip scraping of the second redo record for any recorded changes made to the second database table of the source database based, at least in part, on accessing the information included in the header of the first transaction log file indicating that the second database is out-of-scope for replication between the source database and the target database.
9. The computer-implemented method of claim 1 , wherein determining that the first change record includes at least one change made to the first database table of the source database that is in-scope for replication between the source database and the target database is based, at least in part on:
determining, by one or more processors, that an object identifier of a table row associated with the change record is linked to an object identifier of the first database designated as being in-scope for replication.
10. A computer program product for asynchronous replication between a source database and a target database, the computer program product comprising one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions including instructions to:
determine, while writing a first change record to a first redo record of a first transaction log file, that the first change record includes at least one change to a first database table of a source database that is in-scope for replication between the source database and a target database;
modify a header of the first transaction log file to include information indicating that the first redo record includes a change made to a database table that is in-scope for replication between the source database and the target database; and
responsive to receiving an instruction to replicate changes made to database tables that are in-scope for replication between the source database and the target database:
determine to scrape the first redo record to extract the change made to the first database table of the source database from the first redo record based, at least in part, on accessing the information included in the header of the first transaction log file indicating that the first redo record includes a change made to a database table that is in-scope for replication between the source database and the target database.
11. The computer program product of claim 10 , further comprising instructions to:
retrieve change records from a memory buffer in sequential time order from when each change record was written to the memory buffer.
12. The computer program product of claim 10 , further comprising instructions to:
scrape the first redo record to extract the change made to the database table of the source database; and
transmit the change extracted from the first redo record to the target database for population with the target database.
13. A computer system for asynchronous replication between a source database and a target database, the computer system comprising:
one or more computer processors;
one or more computer readable storage media;
computer program instructions;
the computer program instructions being stored on the one or more computer readable storage media for execution by the one or more computer processors; and
the computer program instructions including instructions to:
determine, while writing a first change record to a first redo record of a first transaction log file, that the first change record includes at least one change to a first database table of a source database that is in-scope for replication between the source database and a target database;
modify a header of the first transaction log file to include information indicating that the first redo record includes a change made to a database table that is in-scope for replication between the source database and the target database; and
responsive to receiving an instruction to replicate changes made to database tables that are in-scope for replication between the source database and the target database:
determine to scrape the first redo record to extract the change made to the first database table of the source database from the first redo record based, at least in part, on accessing the information included in the header of the first transaction log file indicating that the first redo record includes a change made to a database table that is in-scope for replication between the source database and the target database.