Granted Patent B2

US 12,266,077 · App. 17/120,974 · Granted Apr 1, 2025

Inventors: Sheshera Mysore (Sunderland, MA); Sairam Gurajada (San Jose, CA); Lucian Popa (San Jose, CA); Kun Qian (San Jose, CA); Prithviraj Sen (San Jose, CA)

Assignee: International Business Machines Corporation

G06T3/4046G06N3/08G06N5/025G06F16/215G06F40/295G06N3/042G06N3/044G06N3/045

Abstract

Claims (76)

1. A computer-implemented method comprising:

receiving historical pairs of entities including labels indicating whether the historical pairs of entities match;

determining a set of rules for determining whether a pair of entities are matching using the historical pairs of entities, wherein the set of rules comprises a plurality of conditions, each condition representing an instance where the pair of entities could be matching;

building and training a deep neural network using the historical pairs of entities, wherein similarity functions of the historical pairs of entities are input into an embedding layer of the deep neural network, and the deep neural network learns at least predicate parameters and types of conjunctions for the similarity functions and forms entity resolution rules based on the similarity functions and corresponding predicate parameters and types of conjunctions;

developing, using the deep neural network, an entity resolution model based on the historical pairs of entities and the entity resolution rules, wherein the deep neural network comprise a recurrent neural network, and wherein the recurrent neural network is used to determine similarity metrics for the historical pairs of entities in the entity resolution model and analyze the historical pairs of entities and learn similarity metrics from the analysis;

responsive to entity resolution model being developed, receiving, by the entity resolution model, a new pair of entities; and

applying the entity resolution model to the new pair of entities, wherein applying the entity resolution model comprises:

determining whether the one or more conditions and the entity resolution rules associated with the entity resolution model are satisfied by the new pair of entities; and

categorizing the new pair of entities as matching or not matching, based on whether the one or more rules are satisfied.

2. The method of claim 1 , wherein determining the set of rules comprises:

determining similarity metrics of the historical pairs of entities, wherein the similarity metrics are metrics for determining whether a pair of entities are similar;

determining one or more conditions for each similarity metric of the similarity metrics, wherein the one or more conditions indicate when a similarity metric is a matching metric; and

forming the one or more rules within the set of rules for when entities are matching, the one or more rules formed from the one or more conditions.

3. The method of claim 2 , wherein determining similarity metrics for the historical pairs of entities comprises:

identifying predefined similarity metrics based on the historical pairs of entities, wherein the predefined similarity metrics comprise a conjunction of predicates, the conjunction of predicates including a collection of similarity clauses.

4. The method of claim 3 , wherein determining the similarity metrics for the historical pairs of entities further comprises:

determining whether the predefined similarity metrics are sufficient to form a satisfactory entity resolution model; and

in response to determining that the predefined similarity metrics are not sufficient, learning similarity metrics for the historical pair of entities using a neural network, resulting in learned similarity metrics.

5. The method of claim 2 , wherein determining the one or more conditions comprises:

inputting the similarity metrics into the deep neural network;

learning, from the deep neural network, conjunctions and predicate parameters for each similarity metric; and

forming the one or more conditions for each similarity metric based on the conjunctions and the predicate parameters.

6. The method of claim 2 , wherein forming the one or more rules comprises:

grouping corresponding conditions from the one or more conditions, resulting in condition groups; and

creating a rule for each condition group and each condition not in a condition group.

7. The method of claim 1 , wherein the detecting comprises:

determining a significance weight for each rule from the set of rules, wherein the significance weight indicates how significant each rule is in determining whether a pair of entities is matching.

8. The method of claim 1 , further comprising:

determining a predicted score for the new pair of entities, wherein the predicted score is a predicted amount of similarity between the new pair of entities; and

determining whether the predicted score for the new pair of entities is above a threshold predicted score.

9. The method of claim 1 , wherein the set of rules comprises directly interpretable logical rules.

10. A system having one or more computer processors, the system configured to:

receive historical pairs of entities including labels indicating whether the historical pairs of entities match;

determine a set of rules for determining whether a pair of entities are matching using the historical pairs of entities, wherein the set of rules comprises a plurality of conditions, each condition representing an instance where the pair of entities could be matching;

build and train a deep neural network using the historical pairs of entities, wherein similarity functions of the historical pairs of entities are input into an embedding layer of the deep neural network, and the deep neural network learns at least predicate parameters and types of conjunctions for the similarity functions and forms entity resolution rules based on the similarity functions and corresponding predicate parameters and types of conjunctions

develop, using the deep neural network, an entity resolution model based on the historical pairs of entities and the entity resolution rules, wherein the deep neural network comprise a recurrent neural network, and wherein the recurrent neural network is used to determine similarity metrics for the historical pairs of entities in the entity resolution model and analyze the historical pairs of entities and learn similarity metrics from the analysis;

responsive to entity resolution model being developed, receive, by the entity resolution model, a new pair of entities; and

apply the entity resolution model to the new pair of entities, wherein applying the entity resolution model comprises:

determining whether one or more rules from the set of rules are satisfied for the new pair of entities; and

categorizing the new pair of entities as matching or not matching, based on whether the one or more rules are satisfied.

11. The system of claim 10 , wherein determining the set of rules comprises:

determining similarity metrics of the historical pairs of entities, wherein the similarity metrics are metrics for determining whether a pair of entities are similar;

determining one or more conditions for each similarity metric of the similarity metrics, wherein the one or more conditions indicate when a similarity metric is a matching metric; and

forming the one or more rules within the set of rules for when entities are matching, the one or more rules formed from the one or more conditions.

12. The system of claim 11 , wherein determining similarity metrics for the historical pairs of entities comprises:

identifying predefined similarity metrics based on the historical pairs of entities, wherein the predefined similarity metrics comprise a conjunction of predicates, the conjunction of predicates including a collection of similarity clauses.

13. The system of claim 11 , wherein determining the one or more conditions comprises:

inputting the similarity metrics into the deep neural network;

learning, from the deep neural network, conjunctions and predicate parameters for each similarity metric;

determining one or more parameters for each similarity metric; and

forming the one or more conditions for each similarity metric based on the one or more parameters, the conjunctions, and the predicate parameters.

14. The system of claim 11 , wherein forming the one or more rules comprises:

grouping corresponding conditions from the one or more conditions, resulting in condition groups; and

creating a rule for each condition group and each condition not in a condition group.

15. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a server to cause the server to perform a method, the method comprising:

receiving historical pairs of entities including labels indicating whether the historical pairs of entities match;

determining a set of rules for determining whether a pair of entities are matching using the historical pairs of entities, wherein the set of rules comprises a plurality of conditions, each condition representing an instance where the pair of entities could be matching, and wherein one condition of the plurality of conditions identifies a property that must be equal between the pair of entities in order for those entities to be matching;

building and training a deep neural network using the historical pairs of entities, wherein similarity functions of the historical pairs of entities are input into an embedding layer of the deep neural network, and the deep neural network learns at least predicate parameters and types of conjunctions for the similarity functions and forms entity resolution rules based on the similarity functions and corresponding predicate parameters and types of conjunctions;

developing, using the deep neural network, an entity resolution model based on the historical pairs of entities and the entity resolution rules, wherein the deep neural network comprise a recurrent neural network, and wherein the recurrent neural network is used to determine similarity metrics for the historical pairs of entities in the entity resolution model and analyze the historical pairs of entities and learn similarity metrics from the analysis;

responsive to entity resolution model being developed, receiving, by the entity resolution model, a new pair of entities; and

applying the entity resolution model to the new pair of entities, wherein applying the entity resolution model comprises:

determining whether the one or more conditions and the entity resolution rules associated with the entity resolution model are satisfied by the new pair of entities;

categorizing the new pair of entities as matching or not matching, based on whether the one or more rules are satisfied;

detecting that a rule in the set of rules was not significant during the categorizing; and

removing, in part in response to the detecting, the rule from the entity resolution model, resulting in a sparsified entity resolution model.

16. The computer program product of claim 15 , wherein the detecting comprises:

determining a significance weight for each rule from the set of rules, wherein the significance weight indicates how significant each rule is in determining whether a pair of entities is matching.

17. The computer program product of claim 15 , wherein determining the set of rules comprises:

determining similarity metrics of the historical pairs of entities, wherein the similarity metrics are metrics for determining whether a pair of entities are similar;

determining one or more conditions for each similarity metric of the similarity metrics, wherein the one or more conditions indicate when a similarity metric is a matching metric; and

forming the one or more rules within the set of rules for when entities are matching, the one or more rules formed from the one or more conditions.

18. The computer program product of claim 17 , wherein determining the one or more conditions comprises:

inputting the similarity metrics into the deep neural network;

learning, from the deep neural network, conjunctions and predicate parameters for each similarity metric;

determining one or more parameters for each similarity metric; and

forming the one or more conditions for each similarity metric based on the one or more parameters, the conjunctions, and the predicate parameters.

Assignments (1)

Continuity (1)

References Cited (33)