Granted Patent B1

US 9,449,074 · App. 15/066,970 · Granted Sep 20, 2016

Abstract

Claims (45)

1. A computer system configured to obtain changed data from a data source, the computer system comprising:

one or more hardware computer processors configured to execute software code stored in a tangible storage device in order to:

obtain information indicating a plurality of groupings of data stored in one or more files or databases in a data source, the information indicating a number of data items included in each of the plurality of groupings;

determine a first grouping of the plurality of groupings including one or more data items that have changed by comparing a quantity of data items included in the first grouping and a historical quantity of data items included in a corresponding version of the first grouping;

compare the data items included in the first grouping to corresponding data items of the corresponding version of the first grouping to determine which data items of the first grouping from the data source have changed; and

based on the comparison, identify the changed data items.

2. The system of claim 1 , wherein the plurality of groupings is based on timestamps of respective data items, wherein the timestamps indicate respective times at which data items were last updated.

3. The system of claim 2 , wherein the timestamps of the respective data items include a date and a time at which the respective data items were last updated, and wherein the plurality of groupings is based on only the date of the timestamps of the respective data items.

4. The system of claim 3 , wherein each of the plurality of groupings is associated with a different date.

5. The system of claim 3 , wherein the dates associated with each of the plurality of groupings are within a range of dates.

6. The system of claim 1 , wherein the plurality of groupings is based on a first field of respective data items, the first field configured to provide an uneven distribution of data items included in each of the plurality of groupings.

7. The system of claim 1 , wherein the corresponding version of the first grouping comprises a copy of the data items included in the first grouping at the first time.

8. The system of claim 1 , wherein the one or more hardware computer processors are further configured to execute software code stored in the tangible storage device in order to:

in response to determining that the quantity of data items is higher than the historical quantity of data items:

identify the changed data items as added or updated data items; and

forward the changed data items to the destination system to be stored.

9. The system of claim 1 , wherein the one or more hardware computer processors are further configured to execute software code stored in the tangible storage device in order to:

in response to determining that the quantity of data items is lower than the historical quantity of data items:

identify the changed data items as deleted data items; and

forward the changed data items to the destination system to be removed.

10. The system of claim 1 , wherein the one or more hardware computer processors are further configured to execute software code stored in the tangible storage device in order to:

assign a unique identifier to each of the data items included in the first grouping; and

determine whether a first changed data item of the changed data items is a new data item or an updated data item based on the unique identifier associated with the first changed data item.

11. The system of claim 1 , wherein a data item included in the first grouping is a row in the one or more databases or a line in the one or more files.

12. The system of claim 1 , wherein the one or more hardware computer processors are further configured to execute software code stored in the tangible storage device in order to:

obtain the information indicating the plurality of groupings of the data stored in the one or more files in the data source using a first adapter; and

obtain the information indicating the plurality of groupings of the data stored in the one or more databases in the data source using a second adapter.

13. The system of claim 1 , wherein the one or more hardware computer processors are further configured to execute software code stored in the tangible storage device in order to:

obtain the information indicating the plurality of groupings of the data at an interval.

14. The system of claim 1 , wherein the one or more hardware computer processors are further configured to execute software code stored in the tangible storage device in order to:

obtain the information indicating the plurality of groupings of the data in response to receiving a request from the destination system.

15. A method of obtaining changed data from a data source, the method comprising:

obtaining, by one or more hardware computer processors, information indicating a plurality of groupings of data stored in one or more files or databases in a data source, the information indicating a number of data items included in each of the plurality of groupings;

determining, by the one or more hardware computer processors, a first grouping of the plurality of groupings including one or more data items that have changed by comparing a quantity of data items included in the first grouping and a historical quantity of data items included in a corresponding version of the first grouping;

comparing, by the one or more hardware computer processors, the data items included in the first grouping to corresponding data items of the corresponding version of the first grouping to determine which data items of the first grouping from the data source have changed; and

based on the comparison, identifying, by the one or more hardware computer processors, the changed data items.

16. The method of claim 15 , wherein the plurality of groupings is based on timestamps of respective data items, wherein the timestamps indicate respective times at which data items were last updated.

17. The method of claim 15 , wherein the plurality of groupings is based on a first field of respective data items, the first field configured to provide an uneven distribution of data items included in each of the plurality of groupings.

18. The method of claim 15 , wherein the corresponding version of the first grouping comprises a copy of the data items included in the first grouping at the first time.

19. The method of claim 15 , wherein a data item included in the first grouping is a row in the one or more databases or a line in the one or more files.

20. A non-transitory computer readable medium comprising instructions configured to cause a computer processor to:

obtain information indicating a plurality of groupings of data stored in one or more files or databases in a data source, the information indicating a number of data items included in each of the plurality of groupings;

determine a first grouping of the plurality of groupings including one or more data items that have changed by comparing a quantity of data items included in the first grouping and a historical quantity of data items included in a corresponding version of the first grouping;

compare the data items included in the first grouping to corresponding data items of the corresponding version of the first grouping to determine which data items of the first grouping from the data source have changed; and

based on the comparison, identify the changed data items.

Assignments (8)