Managing hierarchical data structures for entity matching
A method comprises storing a plurality of accounts and generating a hierarchical data structure from the plurality of accounts by: identifying a first set of accounts and a second set of accounts responsive to the first set of accounts and the second set of accounts satisfying a first set of rules; linking the first set of accounts to a first entity profile and the second set of accounts to a second entity profile; and responsive to the first plurality of profile values and the second plurality of profile values of the second entity profile satisfying a second set of rules, linking the first entity profile and the second entity profile to a group entity profile.
1 . A computer-implemented method for grouping a plurality of account records, the method comprising:
storing, by at least one processor, a plurality of account records, each account record comprising at least a name, address, and account identifier;
partitioning a set of grouping rules into at least two subsets according to attribute type;
applying, by the at least one processor, a different normalization algorithm to each rule subset according to a corresponding attribute type;
identifying, for each account record, other account records that satisfy at least one grouping rule subset after normalization;
synthesizing identified account records into a group based on the identified account records sharing a common account record that satisfies grouping rule subsets corresponding to different attribute types;
comparing a number of account records identified as satisfying at least one grouping rule subset to a predefined threshold for the at least one grouping rule subset, and responsive to the number exceeding the predefined threshold, removing group associations from the identified account records;
linking the account records in each synthesized group to an entity profile by inserting an entity identifier into an entity attribute-value pair of each account record;
assigning each identified group a group identifier; and
updating each account record in an identified group to include the corresponding group identifier in a group attribute-value pair of each account record, whereby group membership is verifiable by examination of the records.
2 . The method of claim 1 , wherein the normalization algorithm applied to rules related to the name attribute comprises phonetic hashing or edit distance matching.
3 . The method of claim 1 , wherein the rule subset assigned to address attributes applies a minimum similarity threshold based on edit distance or substring matching.
4 . The method of claim 1 , further comprising, upon detecting a change to any attribute in any account record, re-applying only affected rule subsets to update group assignments and corresponding group identifiers.
5 . The method of claim 1 , further comprising, upon determining that any group exceeds a predefined group size threshold, removing the group identifier from all account records of said group.
6 . The method of claim 1 , wherein each group identifier is a globally unique alphanumeric code generated by hashing normalized values of the attributes on which grouping is based.
7 . The method of claim 1 , further comprising storing, for each group identifier, metadata indicating date of creation and number of account records associated with the group.
8 . The method of claim 1 , wherein the account records and group identifiers are maintained within a relational database, with the group identifier corresponding to a foreign key column.
9 . The method of claim 1 , further comprising, prior to group assignment, applying at least one exclusion rule requiring absence of a specified attribute value to prevent an account record from joining a group.
10 . The method of claim 1 , wherein group membership for any account record is detectable by querying the account records for a presence of the group identifier.
11 . A system for grouping account records, comprising:
a memory storing a plurality of account records, each account record comprising at least a name, address, and account identifier; and at least one processor configured to:
partition a set of grouping rules into at least two subsets according to attribute type;
apply a different normalization algorithm to each rule subset according to a corresponding attribute type;
identify, for each account record, other account records that satisfy at least one grouping rule subset after normalization;
synthesize identified account records into a group based on the identified account records sharing a common account record that satisfies grouping rule subsets corresponding to different attribute types;
compare a number of account records identified as satisfying at least one grouping rule subset to a predefined threshold for the at least one grouping rule subset, and responsive to the number exceeding the predefined threshold, remove group associations from the identified account records;
link the account records in each synthesized group to an entity profile by inserting an entity identifier into an entity attribute-value pair of each account record;
assign each identified group a group identifier; and
update each account record in an identified group to include the corresponding group identifier in a group attribute-value pair of each account record, whereby group membership is verifiable by examination of the account records.
12 . The system of claim 11 , wherein the at least one processor is further configured such that the normalization algorithm applied to rules related to the name attribute comprises phonetic hashing or edit distance matching.
13 . The system of claim 11 , wherein the at least one processor is further configured such that the rule subset assigned to address attributes applies a minimum similarity threshold based on edit distance or substring matching.
14 . The system of claim 11 , wherein the at least one processor is further configured, upon detecting a change to any attribute in any account record, to re-apply only affected rule subsets to update group assignments and corresponding group identifiers.
15 . The system of claim 11 , wherein the at least one processor is further configured, upon determining that any group exceeds a predefined group size threshold, to remove the group identifier from all account records of said group.
16 . A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the processor to perform a method comprising:
storing a plurality of account records, each account record comprising at least a name, address, and account identifier;
partitioning a set of grouping rules into at least two subsets according to attribute type;
applying a different normalization algorithm to each rule subset according to a corresponding attribute type;
identifying, for each account record, other account records that satisfy at least one grouping rule subset after normalization;
synthesizing identified account records into a group based on the identified account records sharing a common account record that satisfies grouping rule subsets corresponding to different attribute types;
comparing a number of account records identified as satisfying at least one grouping rule subset to a predefined threshold for the at least one grouping rule subset, and responsive to the number exceeding the predefined threshold, removing group associations from the identified account records;
linking the account records in each synthesized group to an entity profile by inserting an entity identifier into an entity attribute-value pair of each account record;
assigning each identified group a group identifier; and
linking the account records in each synthesized group to an entity profile by inserting an entity identifier into an entity attribute-value pair of each account record.
17 . The non-transitory computer-readable storage medium of claim 16 , wherein the instructions for applying a normalization algorithm to rules related to the name attribute cause the processor to perform phonetic hashing or edit distance matching.
18 . The non-transitory computer-readable storage medium of claim 16 , wherein the instructions for the rule subset assigned to address attributes cause the processor to apply a minimum similarity threshold based on edit distance or substring matching.
19 . The non-transitory computer-readable storage medium of claim 16 , wherein the instructions further cause the processor, upon detecting a change to any attribute in any account record, to re-apply only affected rule subsets to update group assignments and corresponding group identifiers.
20 . The non-transitory computer-readable storage medium of claim 16 , wherein the instructions further cause the processor, upon determining that any group exceeds a predefined group size threshold, to remove the group identifier from all account records of said group.