Method and apparatus for identifying and resolving conflicting data records
A method and apparatus for identifying and resolving conflicting data records are disclosed. The individual data fields of a master record are compared with the corresponding data fields of each source record in a particular data set. For each, one of various matching algorithms is used to assign a field matching score indicating the extent to which the data in the two data fields matches. The particular algorithm used to determine the extent of a match and to assign the corresponding score is dependent on the type of the data field. Once all of the data fields for a particular source record have been analyzed, the sum of the field matching scores is tallied to determine an overall record matching score for that particular source record.
1. A method for identifying matching records in two record sets, the method comprising:
initiating a synchronization process over a computer network;
responsive thereto, analyzing a first record set to match records of the first record set with records of a second record set maintained remotely from the first record set;
selecting a subset of fields shared in common by the records of the two record sets to determine a record match;
for each pairing of a subject record from the first record set and a subject record from the second record set, calculating, based on the subset of fields selected, a record matching score, wherein said each pairing of a subject record from the first record set and a subject record from the second record set comprises a record pair, the record matching scores are calculated based on how well individual fields of the subset of fields in each subject record pair compared, the individual fields of the subset of fields in each subject record pair comprise field group pairings, and for each field group pairing, the individual fields of the subset of fields in each subject record pair are concatenated to form strings;
identifying the matching records of the first record set and the second record set by identifying record pairs having a highest record matching score over a threshold;
excluding identified matched record pairs from further analysis, and successively identifying further matching record pairs according to associated record matching scores of the further matching record pairs and excluding the further matching record pairs from further analysis until all record pairs having record matching scores exceeding the threshold are identified, and
for each pair of the strings representing each field group pairing, calculating a field group matching score, the field group matching scores each comprising a uniqueness component, a difference component, a magnitude component and a conflict score.
2. The method of claim 1 , wherein the synchronization process is initiated at a handset.
3. The method of claim 1 , wherein the synchronization process is initiated at a mobile device.
4. The method of claim 1 , wherein the synchronization process is initiated at a source record device.
5. The method of claim 1 , wherein the synchronization process is initiated according to a schedule.
6. The method of claim 1 , wherein the subset of fields in each subject record pair comprises fields of records of the two record sets most likely to indicate whether individual records of the two record sets match one another.
7. The method of claim 1 , wherein prior to the identifying of the matching records of the first record set and the second record set, scores of each component of each record of each record pair are normalized before the record matching scores of said each record pairs are derived.
8. The method of claim 1 , wherein the individual fields are grouped according to field grouping criteria.
9. The method of claim 1 , wherein concatenation order is determined according to byte-level comparisons of the individual fields.
10. The method of claim 1 , wherein the uniqueness component, the difference component and the magnitude component of each field group matching score is determined on a basis of a Levenshtein Distance between subject ones of the strings undergoing comparison.
11. The method of claim 1 , further comprising combining the field group matching scores for each pair of the field group pairings to determine the record matching score for each pairing of a subject record from the first record set and a subject record from the second record set.
12. The method of claim 11 , wherein the field group matching scores are weighted prior to being combined.
13. The method of claim 2 , wherein weighting of the field group matching scores is determined according to which field group pairings are deemed to be most suggestive of a record match.
14. The method of claim 1 , wherein a score of the uniqueness component is determined based on a Levenshtein Distance as a percentage of a longer one of character arrays comprising the strings.
15. The method of claim 1 , wherein the difference component compensates for any difference between two strings that is solely attributable to a longer length of a long one of the strings.
16. The method of claim 1 wherein the magnitude component takes into consideration an absolute difference in a size of two strings being compared.
17. The method of claim 1 , wherein the conflict score is calculated on a field by field basis.
18. The method of claim 17 , wherein the conflict score is calculated by combining calculated scores of the uniqueness component, the difference component and the magnitude component for each individual field group pairing.