Systems and methods for merging source records in accordance with survivorship rules
View Patent ↗According to some embodiments, a plurality of source records may be received from a plurality of data sources, with each source record including a plurality of fields. It may be determined that a match group of source records from different data sources relate to the same entity, and a single best record may be automatically created for the match group based on field values from different source records in the match group. The creating may includes, for example, assigning a first set of fields to a first survivorship group associated with a first survivorship rule and a second set of fields to a second survivorship group associated with a second survivorship rule. All records in the match group may then be simultaneously ranked in accordance with the first and second survivorship rules using a single query. The best record could then be stored for subsequent use by other applications.
1. A computer implemented method, comprising:
receiving a plurality of source records from a plurality of data sources, each source record including a plurality of fields;
determining that a match group of source records from different data sources relate to the same entity;
automatically creating a single best new record for the match group based on field values from different source records in the match group, wherein said creating includes:
assigning a first set of fields to a first survivorship group associated with a first survivorship rule, the first survivorship rule indicating that the first set of fields are trusted from a first source over a second source,
assigning a second set of fields to a second survivorship group associated with a second survivorship rule, the second survivorship rule indicating that the second set of fields are trusted from the second source over the first source, and
simultaneously ranking all source records in the match group in accordance with the first and second survivorship rules using a single query;
calculating a single source record for each survivorship group based on a plurality of survivorship rule dimensions;
identifying match groups that contain only a single source record, wherein said creating is not automatically performed for the identified match groups; and
storing the single best new record.
2. The method of claim 1 , wherein at least one of the first or second survivorship rule is associated with at least one of: (i) reliability, (ii) recency, or (iii) completeness.
3. The method of claim 1 , wherein at least one of the data sources comprises at least one of: (i) an enterprise resources planning database, (ii) a client relation management database, (iii) a human resources database, or (iv) a legacy database.
4. The method of claim 1 , further comprising:
identifying match groups that contain only a single source record, wherein said creating is not performed for the identified match groups.
5. The method of claim 1 , further comprising:
receiving from a user survivorship configuration information; and
based on the received survivorship configuration information, automatically generating code to, when executed, create the single best new record.
6. The method of claim 5 , wherein the code creates a default survivorship group, an override survivorship group, and the single query constructs the best record simultaneously for multiple match groups.
7. The method of claim 1 , wherein a single source record is calculated for each survivorship group according to a plurality of survivorship rule dimensions.
8. The method of claim 7 , wherein source records are calculated for multiple survivorship groups simultaneously.
9. The method of claim 1 , wherein the query is associated with structured query language protocol.
10. The method of claim 1 , wherein said creating is associated with a column-based in-memory table.
11. A non-transitory, computer-readable medium storing program code instructions executable by a computer processor to perform a method, the method comprising:
receiving a plurality of source records from a plurality of data sources, each source record including a plurality of fields;
determining that a match group of source records from different data sources relate to the same entity;
creating a single best new record for the match group based on field values from different source records in the match group, wherein said creating includes:
assigning a first set of fields to a first survivorship group associated with a first survivorship rule, the first survivorship rule indicating that the first set of fields are trusted from a first source over a second source,
assigning a second set of fields to a second survivorship group associated with a second survivorship rule, the second survivorship rule indicating that the second set of fields are trusted from the second source over the first source, and
simultaneously ranking all source records in the match group in accordance with the first and second survivorship rules using a single query;
calculating a single source record for each survivorship group based on a plurality of survivorship rule dimensions;
identifying match groups that contain only a single source record, wherein said creating is not automatically performed for the identified match groups; and
storing the single best new record.
12. The medium of claim 11 , wherein at least one of the first or second survivorship rule is associated with at least one of: (i) reliability, (ii) recency, or (iii) completeness.
13. The medium of claim 11 , wherein at least one of the data sources comprises at least one of: (i) an enterprise resources planning database, (ii) a client relation management database, (iii) a human resources database, or (iv) a legacy database.
14. The medium of claim 11 , wherein execution of the program code instructions further cause the computer processor to perform:
receiving from a user survivorship configuration information; and
based on the received survivorship configuration information, automatically generating code to, when executed, create the single best new record.
15. The medium of claim 14 , wherein the automatically generated code creates a default survivorship group, an override survivorship group, and the single query constructs the best record simultaneously for multiple match groups.
16. The medium of claim 11 , wherein a single source record is calculated for each survivorship group according to a plurality of survivorship rule dimensions, and further wherein source records are calculated for multiple survivorship groups simultaneously.
17. The medium of claim 11 , wherein the query is associated with structured query language protocol and said creating is associated with a column-based in-memory table.
18. A system, comprising:
a storage device to store a best record data store and a source data store, the source data store to provide a plurality of source records associated with different data sources, each source record including a plurality of fields; and
a processor to (i) receive the source records from the source data store and (ii) store a single best new record into the best record data store, wherein the processor creates a single best record for the match group based on field values from different source records in the match group, wherein said creating includes:
assigning a first set of fields to a first survivorship group associated with a first survivorship rule, the first survivorship rule indicating that first set of fields are trusted from a first source over a second source,
assigning a second set of fields to a second survivorship group associated with a second survivorship rule, the second survivorship rule indicating that the second set of fields are trusted from the second source over the first source, and
simultaneously ranking all source records in the match group in accordance with the first and second survivorship rules using a single query,
calculating a single source record for each survivorship group based on a plurality of survivorship rule dimensions; and
identifying match groups that contain only a single source record, wherein said creating is not automatically performed for the identified match groups.
19. The system of claim 18 , wherein at least one of the first or second survivorship rule is associated with at least one of: (i) reliability, (ii) recency, or (iii) completeness.
20. The system of claim 18 , wherein at least one of the data sources comprises at least one of: (i) an enterprise resources planning database, (ii) a client relation management database, (iii) a human resources database, or (iv) a legacy database.
21. The system of claim 18 , wherein the processor is further to perform:
receiving from a user survivorship configuration information; and
based on the received survivorship configuration information, automatically generating code to, when executed, create the single best new record.
22. The system of claim 21 , wherein the automatically generated code creates a default survivorship group, an override survivorship group, and the single query constructs the best record simultaneously for multiple match groups.
23. The system of claim 18 , wherein a single source record is calculated for each survivorship group according to a plurality of survivorship rule dimensions, and further wherein source records are calculated for multiple survivorship groups simultaneously.
24. The system of claim 18 , wherein the query is associated with structured query language protocol and said creating is associated with a column-based in-memory table.