Method for ensuring referential integrity in replication engines by serializing base level change queue data

1. A computer-implemented method of processing base level data of a change queue in which an order of the data is not serialized at a logical level due to multi-threading in one or more extractors, communication channels or appliers, the base level change queue data being transaction data previously posted to a source database, the method comprising:

(a) providing a logical replication engine which uses multi-threading in the one or more extractors, communication channels or appliers to replicate and post the base level change queue data to a target database in the order that the transaction data is stored in the change queue; and

(b) upon detection in the logical replication engine of base level change queue data in which the order of the base level change queue data is not serialized at the logical level due to the multi-threading in the one or more extractors, communication channels or appliers, serializing the order of the base level change queue data at the logical level before the logical replication engine applies the base level change queue data to the target database, which ensures referential integrity, including resolving required foreign keys, and preserving required intra- and inter-transaction dependencies, in the target database.

2. The method of claim 1 wherein the serializing in step (b) is performed by accumulating index inserts.

3. The method of claim 1 wherein the serializing in step (b) is performed with a set-aside queue.

4. The method of claim 1 wherein the serializing in step (b) is performed with a set-aside queue with transaction end-state processing.

5. The method of claim 1 wherein the change queue is a change log.

6. The method of claim 1 wherein the change queue is an audit trail.

7. The method of claim 1 wherein the change queue is a database of changes.

8. A computer-implemented method of processing base level data of a change queue in which an order of the data is not serialized at a logical level due to multi-threading in one or more extractors, communication channels or appliers, the base level change queue data being transaction data previously posted to a source database, the method comprising:

(a) providing a logical replication engine which uses multi-threading in the one or more extractors, communication channels or appliers to replicate the base level change queue data in which the order of the base level change queue data is not serialized at the logical level to a target database; and

(b) posting the base level change queue data that is not serialized at the logical level due to the multi-threading in the one or more extractors, communication channels or appliers to the target database by asynchronous database access operations, which ensures that subsequent database accesses on the target database are initiated in natural flow order by resolving required foreign keys, and preserving required intra- and inter-transaction dependencies, in the target database.

9. The method of claim 8 wherein step (a) replicates the base level change queue data to the target database in the order that the transaction data is stored in the change queue.

10. The method of claim 8 wherein the asynchronous database access operations use asynchronous calls to the target database.

11. The method of claim 8 wherein the change queue is a change log.

12. The method of claim 8 wherein the change queue is an audit trail.

13. The method of claim 8 wherein the change queue is a database of changes.

14. A computer program product for processing base level data of a change queue in which an order of the data is not serialized at a logical level due to multi-threading in one or more extractors, communication channels or appliers, the base level change queue data being transaction data previously posted to a source database, the computer program product comprising a non-transitory computer-readable storage medium encoded with computer-executable instructions for performing the steps of:

(a) providing a logical replication engine which uses multi-threading in the one or more extractors, communication channels or appliers to replicate and post the base level change queue data to a target database in the order that the transaction data is stored in the change queue; and

(b) upon detection in the logical replication engine of base level change queue data in which the order of the base level change queue data is not serialized at the logical level due to the multi-threading in the one or more extractors, communication channels or appliers, serializing the order of the base level change queue data at the logical level before the logical replication engine applies the base level change queue data to the target database, which ensures referential integrity, including resolving required foreign keys, and preserving required intra- and inter-transaction dependencies, in the target database.

15. The computer program product of claim 14 wherein the serializing in step (b) is performed by accumulating index inserts.

16. The computer program product of claim 14 wherein the serializing in step (b) is performed with a set-aside queue.

17. The computer program product of claim 14 wherein the serializing in step (b) is performed with a set-aside queue with transaction end-state processing.

18. The computer program product of claim 14 wherein the change queue is a change log.

19. The computer program product of claim 14 wherein the change queue is an audit trail.

20. The computer program product of claim 14 wherein the change queue is a database of changes.

21. A computer program product for processing base level data of a change queue in which an order of the data is not serialized at a logical level due to multi-threading in one or more extractors, communication channels or appliers, the base level change queue data being transaction data previously posted to a source database, the computer program product comprising a non-transitory computer-readable storage medium encoded with computer-executable instructions for performing the steps of:

(a) providing a logical replication engine which uses multi-threading in the one or more extractors, communication channels or appliers to replicate the base level change queue data in which the order of the base level change queue data is not serialized at the logical level to a target database; and

(b) posting the base level change queue data that is not serialized at the logical level due to the multi-threading in the one or more extractors, communication channels or appliers to the target database by asynchronous database access operations, which ensures that subsequent database accesses on the target database are initiated in natural flow order by resolving required foreign keys, and preserving required intra- and inter-transaction dependencies, in the target database.

22. The computer program product of claim 21 wherein step (a) replicates the base level change queue data to the target database in the order that the transaction data is stored in the change queue.

23. The computer program product of claim 21 wherein the asynchronous database access operations use asynchronous calls to the target database.

24. The computer program product of claim 21 wherein the change queue is a change log.

25. The computer program product of claim 21 wherein the change queue is an audit trail.

26. The computer program product of claim 21 wherein the change queue is a database of changes.