Optimized statement caching for transaction replay
Transactions are replicated from a source database to a target database in a data replication system. The system includes a first memory location that persistently stores statement text associated with transactions to be replicated. Transactions to be replicated are cached by separating the transactions into statement text and statement data. A determination is made as to whether the statement text has been previously stored in the first memory location. If not, then the statement text is persistently stored to the first memory location. A similar scheme is provided for replicating I/O operations.
1. A method of processing statement text and statement data of a transaction in a process for replicating I/O operations of the transaction from a source database to a target database in a data replication system that includes (i) a change queue, and (ii) a replication engine for use in replicating the I/O operations from the source database to the target database, wherein a statement cache caches statement text associated with I/O operations to be replicated and a first memory location persistently stores statement text associated with I/O operations to be replicated, the method comprising:
(a) caching I/O operations of the transaction to be replicated in the statement cache by separating the I/O operations into:
(i) statement text, and
(ii) statement data;
(b) determining if the statement text has been previously stored in the first memory location;
(c) the data replication system replicating the statement text only if it is determined that the statement text is not previously stored in the first memory location, the data replication system thereby avoiding the need to re-replicate the statement text if the statement text has been previously stored in the first memory location;
(d) persistently storing the statement text associated with the replicated I/O operations in the first memory location whenever the statement text is replicated;
(e) storing the statement data and statement text in a change queue associated with the source database, wherein the I/O operations to be replicated are obtained from the change queue associated with the source database; and
(f) the replication engine obtaining I/O operations from the change queue associated with the source database.
2. The method of claim 1 further including a second memory location for persistently storing the statement data.
3. The method of claim 1 wherein the first memory location is on disk.
4. The method of claim 1 wherein the first memory location is at the target database.
5. An apparatus for processing statement text and statement data of a transaction in a process for replicating I/O operations of the transaction from a source database to a target database in a data replication system that includes (i) a change queue, and (ii) a replication engine for use in replicating the I/O operations from the source database to the target database, wherein a statement cache caches statement text associated with I/O operations to be replicated and a first memory location persistently stores statement text associated with I/O operations to be replicated, the apparatus comprising:
(a) means for caching I/O operations of the transaction to be replicated in the statement cache by separating the I/O operations into:
(i) statement text, and
(ii) statement data;
(b) a data replication system that replicates the statement text only if it is determined that the statement text is not previously stored in the first memory location, the data replication system thereby avoiding the need to re-replicate the statement text if the statement text has been previously stored in the first memory location;
(c) means for persistently storing the statement text associated with the replicated I/O operations in the first memory whenever the statement text is replicated;
(d) means for storing the statement data and statement text in a change queue associated with the source database, wherein the I/O operations to be replicated are obtained from the change queue associated with the source database; and
(e) means for the replication engine to obtain I/O operations from the change queue associated with the source database.
6. The apparatus of claim 5 further including a second memory location for persistently storing the statement data.
7. The apparatus of claim 5 wherein the first memory location is on disk.
8. The apparatus of claim 5 wherein the first memory location is at the target database.
9. The method of claim 1 further comprising:
(g) assigning a statement identifier to statement text, wherein the statement identifier is used in place of the statement text.
10. The apparatus of claim 5 further comprising:
(g) means for assigning a statement identifier to statement text, wherein the statement identifier is used in place of the statement text.
11. The method of claim 2 wherein the second memory location is on disk.
12. The method of claim 2 wherein the second memory location is at the target database.
13. The apparatus of claim 6 wherein the second memory location is on disk.
14. The apparatus of claim 6 wherein the second memory location is at the target database.