System and method for genealogical entity resolution
Systems, methods, and other techniques for genealogical entity resolution. In some embodiments, first tree data and second tree data are obtained, the first tree data corresponding to a first tree person and the second tree data corresponding to a second tree person. A set of features is extracted from the first tree data and the second tree data. An individual-level similarity score for each possible pairing of tree persons is generated based on the set of features. A set of most-similar tree persons is identified based on the individual-level similarity score for each possible pairing. A plurality of individual-level similarity vectors for the set of most-similar tree persons are provided as input to a family-level ML model to determine that the first tree person and the second tree person correspond to a same individual.
1 . A computer-implemented method comprising:
extracting, from a first tree node of a first data tree comprising a first plurality of interconnected nodes representing relationships among a first set of tree persons, a first set of features using a feature extractor;
extracting, from a second tree node of a second data tree comprising a second plurality of interconnected nodes representing relationships among a second set of tree persons, a second set of features using the feature extractor;
identifying, within a cluster database storing clusters of tree nodes and from the first set of features and the second set of features, a cluster comprising a set of tree nodes including tree nodes from the first data tree and the second data tree, wherein the set of tree nodes corresponds to a single tree person by:
generating an individual-level similarity score between the first tree node and the second tree node;
generating additional individual-level similarity scores between additional nodes of the first plurality of interconnected nodes within the first data tree and additional nodes of the second plurality of interconnected nodes within the first data tree and additional nodes of the second plurality of interconnected nodes within the second data tree; and
determining the cluster based on the individual-level similarity score and the additional individual-level similarity scores; and
based on identifying the cluster comprising the set of tree nodes corresponding to the single tree person, modifying the cluster database to include the cluster representing the single tree person.
2 . The computer-implemented method of claim 1 , wherein modifying the cluster database comprises resolving a duplicate tree person by:
generating a new node connection between the first tree node and the second tree node to merge the first tree node and the second tree node within the cluster of the cluster database; or
generating a new cluster in the cluster database, wherein the new cluster includes a node connection between the first tree node and the second tree node based on identifying the cluster comprising the set of tree nodes corresponding to the single tree person.
3 . The computer-implemented method of claim 1 , wherein identifying the cluster comprises grouping the first tree node and the second tree node together based on the first set of features and the second set of features corresponding to the single tree person.
4 . The computer-implemented method of claim 2 , wherein identifying the cluster comprises:
determining the cluster based on the individual-level similarity score and the additional individual-level similarity scores by utilizing an individual-level machine learning model.
5 . The computer-implemented method of claim 1 , further comprising:
generating a family-level similarity score between the first tree node of the first data tree and the second tree node of the second data tree, wherein the family-level similarity score indicates a similarity of the first set of features and the second set of features across familial relationships; and
identifying the cluster based on the family-level similarity score.
6 . The computer-implemented method of claim 5 , further comprising:
generating a match classification utilizing a family-level machine learning model to process an aggregated set of features from the first set of features and the second set of features; and
identifying the cluster based on the match classification.
7 . The computer-implemented method of claim 1 , wherein identifying the cluster comprises using a most-similar relationships identifier to compare similarity scores between nodes in the first plurality of interconnected nodes and the second plurality of interconnected nodes.
8 . A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
extracting, from a first tree node of a first data tree comprising a first plurality of interconnected nodes representing relationships among a first set of tree persons, a first set of features using a feature extractor;
extracting, from a second tree node of a second data tree comprising a second plurality of interconnected nodes representing relationships among a second set of tree persons, a second set of features using the feature extractor;
identifying, within a cluster database storing clusters of tree nodes and from the first set of features and the second set of features, a cluster comprising a set of tree nodes including tree nodes from the first data tree and the second data tree, wherein the set of tree nodes corresponds to a single tree person by:
generating an individual-level similarity score between the first tree node and the second tree node;
generating additional individual-level similarity scores between additional nodes of the first plurality of interconnected nodes within the first data tree and additional nodes of the second plurality of interconnected nodes within the first data tree and additional nodes of the second plurality of interconnected nodes within the second data tree; and
determining the cluster based on the individual-level similarity score and the additional individual-level similarity scores; and
based on identifying the cluster comprising the set of tree nodes corresponding to the single tree person, modifying the cluster database to include the cluster representing the single tree person.
9 . The non-transitory computer-readable medium of claim 8 , wherein modifying the cluster database comprises resolving a duplicate tree person by:
generating a new node connection between the first tree node and the second tree node to merge the first tree node and the second tree node within the cluster of the cluster database; or
generating a new cluster in the cluster database, wherein the new cluster includes a node connection between the first tree node and the second tree node based on identifying the cluster comprising the set of tree nodes corresponding to the single tree person.
10 . The non-transitory computer-readable medium of claim 8 , wherein identifying the cluster comprises grouping the first tree node and the second tree node together based on the first set of features and the second set of features corresponding to the single tree person.
11 . The non-transitory computer-readable medium of claim 9 , wherein identifying the cluster comprises:
determining the cluster based on the individual-level similarity score and the additional individual-level similarity scores by utilizing an individual-level machine learning model.
12 . The non-transitory computer-readable medium of claim 8 , further comprising:
generating a family-level similarity score between the first tree node of the first data tree and the second tree node of the second data tree, wherein the family-level similarity score indicates a similarity of the first set of features and the second set of features across familial relationships; and
identifying the cluster based on the family-level similarity score.
13 . The non-transitory computer-readable medium of claim 12 , further comprising:
generating a match classification utilizing a family-level machine learning model to process an aggregated set of features from the first set of features and the second set of features; and
identifying the cluster based on the match classification.
14 . The non-transitory computer-readable medium of claim 8 , wherein identifying the cluster comprises using a most-similar relationships identifier to compare similarity scores between nodes in the first plurality of interconnected nodes and the second plurality of interconnected nodes.
15 . A system comprising:
one or more processors; and
a computer-readable medium comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
extracting, from a first tree node of a first data tree comprising a first plurality of interconnected nodes representing relationships among a first set of tree persons, a first set of features using a feature extractor;
extracting, from a second tree node of a second data tree comprising a second plurality of interconnected nodes representing relationships among a second set of tree persons, a second set of features using the feature extractor;
identifying, within a cluster database storing clusters of tree nodes and from the first set of features and the second set of features, a cluster comprising a set of tree nodes including tree nodes from the first data tree and the second data tree, wherein the set of tree nodes corresponds to a single tree person by:
generating an individual-level similarity score between the first tree node and the second tree node;
generating additional individual-level similarity scores between additional nodes of the first plurality of interconnected nodes within the first data tree and additional nodes of the second plurality of interconnected nodes within the first data tree and additional nodes of the second plurality of interconnected nodes within the second data tree; and
determining the cluster based on the individual-level similarity score and the additional individual-level similarity scores; and
based on identifying the cluster comprising the set of tree nodes corresponding to the single tree person, modifying the cluster database to include the cluster representing the single tree person.
16 . The system of claim 15 , wherein modifying the cluster database comprises resolving a duplicate tree person by:
generating a new node connection between the first tree node and the second tree node to merge the first tree node and the second tree node within the cluster of the cluster database; or
generating a new cluster in the cluster database, wherein the new cluster includes a node connection between the first tree node and the second tree node based on identifying the cluster comprising the set of tree nodes corresponding to the single tree person.
17 . The system of claim 15 , wherein identifying the cluster comprises grouping the first tree node and the second tree node together based on the first set of features and the second set of features corresponding to the single tree person.
18 . The system of claim 16 , wherein identifying the cluster comprises:
determining the cluster based on the individual-level similarity score and the additional individual-level similarity scores by utilizing an individual-level similarity score machine learning model.
19 . The system of claim 15 , further comprising:
generating a family-level similarity score between the first tree node of the first data tree and the second tree node of the second data tree, wherein the family-level similarity score indicates a similarity of the first set of features and the second set of features across familial relationships; and
identifying the cluster based on the family-level similarity score.
20 . The system of claim 19 , further comprising:
generating a match classification utilizing a family-level machine learning model to process an aggregated set of features from the first set of features and the second set of features; and
identifying the cluster based on the match classification.