IP Library › Granted Patent US 11,200,208
Granted Patent B2
US 11,200,208 · App. 16/738,896 · Granted Dec 14, 2021

Removing non-deterministic behavior in a change data capture merge

Inventor: Alejandro Humberto Abdelnur (Barcelona, ES)
Assignee: StreamSets, Inc.
G06F16/178G06F16/214
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,200,208
App. No.
16/738,896
Granted
Dec 14, 2021
Kind
B2
Abstract

Systems and methods herein describe accessing an original change data capture (CDC) dataset comprising information describing changes to a source database, the original CDC dataset comprising a plurality of entries; identifying a first entry of the plurality of entries comprising a primary-key, a first operation and entry data; identifying a set of entries in the plurality of entries that includes the primary-key; comparing the first operation of the first entry with a second operation of a second entry in the set of entries; updating the first operation and the entry data based on the comparison; generating a new entry based on the updating of the first operation and the entry data; storing the new entry in a consolidated CDC dataset; and applying the consolidated CDC dataset to a target database.

Claims (49)

1. A method comprising:

accessing an original change data capture (CDC) dataset comprising information describing changes to a source database, the original CDC dataset comprising a plurality of entries;

identify a first entry in the plurality of entries, the first entry comprising a primary-key, a first operation and entry data;

identifying a set of entries in the plurality of entries that includes the primary-key;

removing, the first entry and the set of entries from the original CDC dataset;

comparing the first operation of the first entry with a second operation of a second entry in the set of entries;

updating the first operation and the entry data based on the comparison;

generating a new entry based on the updating of the first operation and the entry data;

storing the new entry in a consolidated CDC dataset; and

synchronizing a target database with the source database by applying the consolidated CDC dataset to the target database.

2. The method of claim 1 , wherein applying the consolidated CDC dataset to the target dataset is in response to determining the original CDC dataset is empty.

3. The method of claim 1 , wherein the first entry further comprises an entity.

4. The method of claim 1 , wherein each entry in the consolidated CDC dataset has a unique primary-key.

5. The method of claim 1 , wherein the first operation of the first entry describes a type of change applied to the entry data.

6. The method of claim 1 , wherein the identified set of entries in the plurality of entries is an ordered list.

7. The method of claim 6 , wherein the ordered list matches the order the changes were performed in the source database.

8. A computing apparatus comprising:

a processor; and

a memory storing instructions that, when executed by the processor, configure the apparatus to perform operations comprising:

accessing an original change data capture (CDC) dataset comprising information describing changes to a source database, the original CDC dataset comprising a plurality of entries;

identify a first entry in the plurality of entries, the first entry comprising a primary-key, a first operation and entry data;

identifying a set of entries in the plurality of entries that includes the primary-key;

removing, the first entry and the set of entries from the original CDC dataset;

comparing the first operation of the first entry with a second operation of a second entry in the set of entries;

updating the first operation and the entry data based on the comparison;

generating a new entry based on the updating of the first operation and the entry data;

storing the new entry in a consolidated CDC dataset; and

synchronizing a target database with the source database by applying the consolidated CDC dataset to the target database.

9. The computing apparatus of claim 8 , wherein applying the consolidated CDC dataset to the target dataset is in response to determining the original CDC dataset is empty.

10. The computing apparatus of claim 8 , wherein the first entry further comprises an entity.

11. The computing apparatus of claim 8 , wherein each entry in the consolidated CDC dataset has a unique primary-key.

12. The computing apparatus of claim 8 , wherein the first operation of the first entry describes a type of change applied to the entry data.

13. The computing apparatus of claim 8 , wherein the identified set of entries in the plurality of entries is an ordered list.

14. The computing apparatus of claim 13 , wherein the ordered list matches the order the changes were performed in the source database.

15. A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to perform operations comprising:

accessing an original change data capture (CDC) dataset comprising information describing changes to a source database, the original CDC dataset comprising a plurality of entries;

identify a first entry in the plurality of entries, the first entry comprising a primary-key, a first operation and entry data;

identifying a set of entries in the plurality of entries that includes the primary-key;

removing, the first entry and the set of entries from the original CDC dataset;

comparing the first operation of the first entry with a second operation of a second entry in the set of entries;

updating the first operation and the entry data based on the comparison;

generating a new entry based on the updating of the first operation and the entry data;

storing the new entry in a consolidated CDC dataset; and

synchronizing a target database with the source database by applying the consolidated CDC dataset to the target database.

16. The computer-readable storage medium of claim 15 , wherein applying the consolidated CDC dataset to the target dataset is in response to determining the original CDC dataset is empty.

17. The computer-readable storage medium of claim 15 , wherein the first entry further comprises an entity.

18. The computer-readable storage medium of claim 15 , wherein each entry in the consolidated CDC dataset has a unique primary-key.

19. The computer-readable storage medium of claim 15 , wherein the identified set of entries in the plurality of entries is an ordered list.

20. The computer-readable storage medium of claim 19 , wherein the ordered list matches the order the changes were performed in the source database.

Assignments (4)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 8, 2024
From: STREAMSETS, INC.
To: INTERNATIONAL BUSINESS MACHINES CORPORATION
Reel/Frame 069219/0116 →
RELEASE OF SECURITY INTEREST Recorded May 9, 2022
From: AB PRIVATE CREDIT INVESTORS LLC, AS ADMINISTRATIVE AGENT
To: STREAMSETS, INC.
Reel/Frame 059870/0831 →
SECURITY INTEREST Recorded Nov 25, 2020
From: STREAMSETS, INC.
To: AB PRIVATE CREDIT INVESTORS LLC, AS ADMINISTRATIVE AGENT
Reel/Frame 054472/0345 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 5, 2020
From: ABDELNUR, ALEJANDRO HUMBERTO
To: STREAMSETS, INC.
Reel/Frame 051725/0860 →
Continuity (1)
Related Publication 20210216504A1 · Jul 15, 2021