Machine-assisted object matching
Embodiments are directed towards managing data using modeling platform. A principal data set and match data set may be provided. The principal data set may include principal objects and the match data set may include other model objects. Blocking may associate each of the plurality of principal objects with potential match objects. Principal objects from the plurality of principal objects may be selected based on heuristics, their associated potential match objects, or the like. Potential match objects may be labeled as a true match or a non-match with respect to their associated principal object. A ranker may be trained to identify top-ranked matches based on the labeled potential match objects. The trained ranker may be employed to indicate matches where each matching other model object is a potential match object that is ranked higher than other potential match objects that are associated with its matched principal object.
1. A method for managing data using one or more processors, included in one or more network computers, to execute a modeling platform server that performs actions, comprising:
employing a source data server to provide a principal data set and another data set, wherein the principal data set includes a plurality of principal objects and the other data set includes a plurality of other objects;
instantiating a first engine to perform actions including:
associating each of the plurality of principal objects with one or more other objects that are selected from the plurality of other objects, wherein the one or more other objects are selected as potential matches to one or more of the principal objects;
selecting one or more principal objects of the plurality of principal objects based on one or more heuristics and the one or more selected principal objects association with selected other objects; and
labeling one or more of the selected other objects as either a match or a non-match of their associated principal object; and
instantiating a second engine to perform actions, including:
training a ranker to identify a selected other object that is top-ranked in similarity to its association with the one or more principal objects based on the labeling of the one or more selected other objects; and
employing the trained ranker to indicate a match among the plurality of principal objects and the plurality of other objects; and
instantiating a third engine to selectively filter the plurality of other objects to rank each matched other object higher than other objects associated with a same principal object, wherein geolocation information from a GPS device is employed to localize a display of the ranked other objects including one or more of time zone or language, and wherein a hardware security module is used to provide tamper resistant safeguarding of cryptographic information.
2. The method of claim 1 , further comprising:
providing two or more data sets that include raw data; and
transforming the raw data into a plurality of model objects that are included in one or more of the principal data set or the other data set, wherein each model object has the same type and number of features.
3. The method of claim 1 , further comprising:
employing the first engine to train a secondary classifier to distinguish between matches and non-matches based on the indicated matches; and
employing the secondary classifier to discover incorrect indications of matches that were provided by the second engine.
4. The method of claim 1 , further comprising:
grouping the other objects in the other data set with potential duplicate objects in the other data set;
labeling one or more duplicate objects included in the other data set as matches or non-matches;
training another ranker to rank potential duplicate objects based on their similarity to their associated principal duplicate object;
employing the other ranker to rank potential duplicate objects based on their similarity to their associated principal duplicate object, wherein the top-ranked potential duplicate object is indicated as a duplicate object;
training another secondary classifier to discover duplicate objects that are falsely indicated by the other ranker;
discovering duplicate model objects in the other data set based on the other ranker and the other secondary classifier; and
removing the duplicate model objects from the plurality of other objects.
5. The method of claim 1 , wherein labeling the one or more selected other objects, further comprises:
communicating each of the one or more principal objects and their associated selected other objects to a user; and
displaying each of the one or more principal objects and their associated selected other objects to the user; and
enabling the user to identify a match with the one or more principal data objects from among the displayed selected other objects.
6. The method of claim 1 , wherein providing the one or more principal objects further comprises, selecting the principal model objects at random from a portion of the plurality of principal model objects, wherein the portion of principal model objects are associated with two or more selected other objects and one or more of the two or more selected other objects have a similarity score that exceeds a threshold value.
7. The method of claim 1 , further comprising, employing the first engine to provide a similarity score for each of the selected other objects, wherein the similarity score is arranged to indicate a measure of feature-based similarity between principal objects and their associated selected other objects.
8. The method of claim 1 , further comprising, distributing one or more instances of one or more of the first engine, or the second engine across two or more separate network computers, wherein the one or more instances may execute in parallel or concurrently.
9. A system for managing data, comprising:
a network computer, comprising:
a transceiver that communicates over the network;
a memory that stores at least instructions; and
one or more processor devices that execute instructions that perform actions, including:
providing a principal data set and another data set, wherein the principal data set includes a plurality of principal objects and the other data set includes a plurality of other objects;
instantiating a first engine to perform actions including:
associating each of the plurality of principal objects with one or more other objects that are selected from the plurality of other objects, wherein the one or more other objects are selected as potential matches to one or more of the principal objects;
selecting one or more principal objects of the plurality of principal objects based on one or more heuristics and the one or more selected principal objects association with selected other objects; and
labeling one or more of the selected other objects as either a match or a non-match of their associated principal object; and
instantiating a second engine to perform actions, including:
training a ranker to identify a selected other object that is top-ranked in similarity to its association with the one or more principal objects based on the labeling of the one or more selected other objects; and
employing the trained ranker to indicate a match among the plurality of principal objects and the plurality of other objects; and
instantiating a third engine to selectively filter the plurality of other objects to rank each matched other object higher than other objects associated with a same principal object, wherein geolocation information from a GPS device is employed to localize the displaying of the ranked other objects including one or more of time zone or language, and wherein a hardware security module is used to provide tamper resistant safeguarding of cryptographic information; and
a client computer, comprising:
a client computer transceiver that communicates over the network;
a client computer memory that stores at least instructions; and
one or more processor devices that execute instructions that perform actions, including:
displaying one or more of the one or more principal objects and the one or more selected other objects on a display of the client computer, wherein the displayed selected other objects are associated with a same principal object.
10. The system of claim 9 , further comprising:
providing two or more data sets that include raw data; and
transforming the raw data into a plurality of model objects that are included in one or more of the principal data set or the other data set, wherein each model object has the same type and number of features.
11. The system of claim 9 , further comprising:
employing the first engine to train a secondary classifier to distinguish between matches and non-matches based on the indicated matches; and
employing the secondary classifier to discover incorrect indications of matches that were provided by the second engine.
12. The system of claim 9 , further comprising:
grouping the other objects in the other data set with potential duplicate objects in the other data set;
labeling one or more duplicate objects included in the other data set as matches or non-matches;
training another ranker to rank potential duplicate objects based on their similarity to their associated principal duplicate object;
employing the other ranker to rank potential duplicate objects based on their similarity to their associated principal duplicate object, wherein the top-ranked potential duplicate object is indicated as a duplicate object;
training another secondary classifier to discover duplicate objects that are falsely indicated by the other ranker;
discovering duplicate model objects in the other data set based on the other ranker and the other secondary classifier; and
removing the duplicate model objects from the plurality of other objects.
13. The system of claim 9 , wherein labeling the one or more selected other objects, further comprises:
communicating each of the one or more principal objects and their associated selected other objects to a user; and
displaying each of the one or more principal objects and their associated selected other objects to the user; and
enabling the user to identify a match with the one or more principal data objects from among the displayed selected other objects.
14. The system of claim 9 , wherein providing the one or more principal objects further comprises, selecting the principal model objects at random from a portion of the plurality of principal model objects, wherein the portion of principal model objects are associated with two or more selected other objects and one or more of the two or more selected other objects have a similarity score that exceeds a threshold value.
15. The system of claim 9 , further comprising, employing the first engine to provide a similarity score for each of the selected other objects, wherein the similarity score is arranged to indicate a measure of feature-based similarity between principal objects and their associated selected other objects.
16. The system of claim 9 , further comprising, distributing one or more instances of one or more of the first engine, or the second engine across two or more separate network computers, wherein the one or more instances may execute in parallel or concurrently.
17. A processor readable non-transitory storage media that includes instructions for managing data, wherein execution of the instructions by one or more hardware processors performs actions, comprising:
providing a principal data set and another data set, wherein the principal data set includes a plurality of principal objects and the other data set includes a plurality of other objects;
instantiating a first engine to perform actions including:
associating each of the plurality of principal objects with one or more other objects that are selected from the plurality of other objects, wherein the one or more other objects are selected as potential matches to one or more of the principal objects;
selecting one or more principal objects of the plurality of principal objects based on one or more heuristics and the one or more selected principal objects association with selected other objects; and
labeling one or more of the selected other objects as either a match or a non-match of their associated principal object; and
instantiating a second engine to perform actions, including:
training a ranker to identify a selected other object that is top-ranked in similarity to its association with the one or more principal objects based on the labeling of the one or more selected other objects; and
employing the trained ranker to indicate a match among the plurality of principal objects and the plurality of other objects; and
instantiating a third engine to selectively filter the plurality of other objects to rank each matched other object higher than other objects associated with a same principal object, wherein geolocation information from a GPS device is employed to localize a display of the ranked other objects including one or more of time zone or language, and wherein a hardware security module is used to provide tamper resistant safeguarding of cryptographic information.
18. The media of claim 17 , further comprising:
providing two or more data sets that include raw data; and
transforming the raw data into a plurality of model objects that are included in one or more of the principal data set or the other data set, wherein each model object has the same type and number of features.
19. The media of claim 17 , further comprising:
employing the first engine to train a secondary classifier to distinguish between matches and non-matches based on the indicated matches; and
employing the secondary classifier to discover incorrect indications of matches that were provided by the second engine.
20. The media of claim 17 , further comprising:
grouping the other objects in the other data set with potential duplicate objects in the other data set;
labeling one or more duplicate objects included in the other data set as matches or non-matches;
training another ranker to rank potential duplicate objects based on their similarity to their associated principal duplicate object;
employing the other ranker to rank potential duplicate objects based on their similarity to their associated principal duplicate object, wherein the top-ranked potential duplicate object is indicated as a duplicate object;
training another secondary classifier to discover duplicate objects that are falsely indicated by the other ranker;
discovering duplicate model objects in the other data set based on the other ranker and the other secondary classifier; and
removing the duplicate model objects from the plurality of other objects.
21. The media of claim 17 , wherein labeling the one or more selected other objects, further comprises:
communicating each of the one or more principal objects and their associated selected other objects to a user; and
displaying each of the one or more principal objects and their associated selected other objects to the user; and
enabling the user to identify a match with the one or more principal data objects from among the displayed selected other objects.
22. The media of claim 17 , wherein providing the one or more principal objects further comprises, selecting the principal model objects at random from a portion of the plurality of principal model objects, wherein the portion of principal model objects are associated with two or more selected other objects and one or more of the two or more selected other objects have a similarity score that exceeds a threshold value.
23. The media of claim 17 , further comprising, employing the first engine to provide a similarity score for each of the selected other objects, wherein the similarity score is arranged to indicate a measure of feature-based similarity between principal objects and their associated selected other objects.
24. A network computer for managing data, comprising:
a transceiver that communicates over the network;
a memory that stores at least instructions; and
one or more processor devices that execute instructions that perform actions, including:
providing a principal data set and another data set, wherein the principal data set includes a plurality of principal objects and the other data set includes a plurality of other objects;
instantiating a first engine to perform actions including:
associating each of the plurality of principal objects with one or more other objects that are selected from the plurality of other objects, wherein the one or more other objects are selected as potential matches to one or more of the principal objects;
selecting one or more principal objects of the plurality of principal objects based on one or more heuristics and the one or more selected principal objects association with selected other objects; and
labeling one or more of the selected other objects as either a match or a non-match of their associated principal object; and
instantiating a second engine to perform actions, including:
training a ranker to identify a selected other object that is top-ranked in similarity to its association with the one or more principal objects based on the labeling of the one or more selected other objects; and
employing the trained ranker to indicate a match among the plurality of principal objects and the plurality of other objects; and
instantiating a third engine to selectively filter the plurality of other objects to rank each matched other object higher than other objects associated with a same principal object, wherein geolocation information from a GPS device is employed to localize a display of the ranked other objects including one or more of time zone or language, and wherein a hardware security module is used to provide tamper resistant safeguarding of cryptographic information.
25. The network computer of claim 24 , further comprising:
providing two or more data sets that include raw data; and
transforming the raw data into a plurality of model objects that are included in one or more of the principal data set or the other data set, wherein each model object has the same type and number of features.
26. The network computer of claim 24 , further comprising:
employing the first engine to train a secondary classifier to distinguish between matches and non-matches based on the indicated matches; and
employing the secondary classifier to discover incorrect indications of matches that were provided by the second engine.
27. The network computer of claim 24 , further comprising:
grouping the other objects in the other data set with potential duplicate objects in the other data set;
labeling one or more duplicate objects included in the other data set as matches or non-matches;
training another ranker to rank potential duplicate objects based on their similarity to their associated principal duplicate object;
employing the other ranker to rank potential duplicate objects based on their similarity to their associated principal duplicate object, wherein the top-ranked potential duplicate object is indicated as a duplicate object;
training another secondary classifier to discover duplicate objects that are falsely indicated by the other ranker;
discovering duplicate model objects in the other data set based on the other ranker and the other secondary classifier; and
removing the duplicate model objects from the plurality of other objects.
28. The network computer of claim 24 , wherein labeling the one or more selected other objects, further comprises:
communicating each of the one or more principal objects and their associated selected other objects to a user; and
displaying each of the one or more principal objects and their associated selected other objects to the user; and
enabling the user to identify a match with the one or more principal data objects from among the displayed selected other objects.
29. The network computer of claim 24 , wherein providing the one or more principal objects further comprises, selecting the principal model objects at random from a portion of the plurality of principal model objects, wherein the portion of principal model objects are associated with two or more selected other objects and one or more of the two or more selected other objects have a similarity score that exceeds a threshold value.
30. The network computer of claim 24 , further comprising, employing the first engine to provide a similarity score for each of the selected other objects, wherein the similarity score is arranged to indicate a measure of feature-based similarity between principal objects and their associated selected other objects.