Matching co-referring entities from serialized data for schema inference
View Patent ↗A system and method provide for identifying coreference from serialized data coming from different services. The method includes generating a tree structure from serialized data. The serialized data includes responses to queries from the different services. The responses each identify a hierarchical relationship between a respective set of objects. Nodes of the tree structure each have a name corresponding to a respective one of the objects. The tree structure is traversed in a breadth first manner and, for each node in the tree structure, a respective pairwise similarity is computed with each of the other nodes of the tree structure. The computed pairwise similarity is compared with a threshold to identify co-referring nodes that refer to a same entity. The threshold is a function of a depth of the node in the tree structure.
1. A method for identifying coreference from serialized data comprising:
generating a tree structure from serialized data, the serialized data comprising responses to queries from different services, the responses each identifying a hierarchical relationship between a respective set of objects, nodes of the tree structure each having a name corresponding to a respective one of the objects;
traversing the tree structure in a breadth first manner and for each node in the tree structure, computing a respective pairwise similarity with other nodes of the tree structure;
comparing the computed pairwise similarity with a threshold to identify co-referring nodes that refer to a same entity, the threshold being a function of a depth of the node in the tree structure;
merging two nodes identified as being co-referring nodes, the merging including identifying all of the children of the two nodes as having both of the two nodes as their parents;
generating a directed acyclic graph which includes the merged nodes; and
outputting information based on the identified co-referring nodes, the information comprising the directed acyclic graph or information based thereon,
wherein at least one of the generating of the tree structure, computing a respective pairwise similarity, and identifying co-referring nodes is performed with a processor.
2. The method of claim 1 , wherein the computing of the pairwise similarity comprises computing a first similarity based on a similarity of the two nodes being compared and computing a second similarity based on a similarity of children of the two nodes being compared and aggregating the first and second similarities.
3. The method of claim 2 , wherein the computing of the first similarity is also based on parents of the two nodes being compared.
4. The method of claim 2 , wherein the computing of the second similarity comprises identifying a number of overlapping children of the first and second nodes.
5. The method of claim 4 , wherein the second similarity is a function of:
sim
2
(
n
,
n
′
)
=
n
.
children
⋂
n
′
.
children
min
(
n
.
children
,
n
′
.
children
where n represents the first node and n′ represents the second node.
6. The method of claim 2 , wherein the aggregating comprises multiplying the first and second similarities.
7. The method of claim 6 , wherein the threshold is a function of
m
+
(
M
-
m
)
1
-
h
H
,
where m and M are predefined constant values, h is a depth of the node and H is a maximum depth of the tree structure.
8. The method of claim 1 , wherein the threshold is a concave decreasing function of the depth.
9. The method of claim 1 , further comprising merging two nodes identified as being co-referring objects, the merging including identifying all of the children of the two nodes as having both of the two nodes as their parents.
10. The method of claim 9 , comprising generating a directed acyclic graph which includes the merged nodes.
11. The method of claim 1 wherein the serialization format of the serialized data is JSON.
12. The method of claim 1 , wherein the generating of the tree structure from serialized data comprises merging objects of a list into one node which has as its name the key-value of its parent key.
13. The method of claim 1 , wherein the services are services of a same organization that use at least one of:
different names for the same object, and
a same name for different objects.
14. The method of claim 1 , wherein the serialized data comprises responses to queries of a database, the method further comprising:
enriching a database with information on the identified co-referring nodes.
15. A method for identifying coreference from serialized data comprising:
generating a tree structure from serialized data, the serialized data comprising responses to database queries from different services, the responses each identifying a hierarchical relationship between a respective set of objects, nodes of the tree structure each having a name corresponding to a respective one of the objects;
traversing the tree structure in a breadth first manner and for each node in the tree structure, computing a respective pairwise similarity with other nodes of the tree structure, the computing of the pairwise similarity comprising:
computing a first similarity based on a similarity of the two nodes being compared, and
computing a second similarity based on a similarity of children of the two nodes being compared and aggregating the first and second similarities, wherein at least one of:
a) the computing of the first similarity is also based on parents of the two nodes being compared and comprises computing a similarity based on a maximal value of the name of the second node and a combination of the name of the second node with its parent's name, and
b) the computing of the second similarity comprises identifying a number of overlapping children of the first and second nodes, the second similarity being a function of:
sim
2
(
n
,
n
′
)
=
n
.
children
⋂
n
′
.
children
min
(
n
.
children
,
n
′
.
children
)
where n represents the first node and n′ represents the second node;
comparing the computed pairwise similarity with a threshold to identify co-referring nodes that refer to a same entity, the threshold being a function of a depth of the node in the tree structure; and
enriching the queried database with information on the identified co-referring nodes,
wherein at least one of the generating of the tree structure, computing a respective pairwise similarity, and identifying co-referring nodes is performed with a processor.
16. A computer program product comprising a non-transitory recording medium storing instructions, which when executed on a computer, cause the computer to perform the method of claim 1 .
17. A system comprising memory which stores instructions for performing the method of claim 1 and a processor in communication with the memory which executes the instructions.
18. A system for identifying co-referring objects from serialized data comprising:
a querying component which receives responses from a plurality of different services that query a database, each of the responses comprising serialized data in the form of a respective sub-tree, the responses each identifying a hierarchical relationship between a respective set of objects;
an inference component which generates a single tree structure by connecting the serialized data sub-trees, nodes of the tree structure each having a name corresponding to a respective one of the objects;
a matching component which traverses the tree structure in a breadth first manner and for each considered node in the tree structure, computes a respective pairwise similarity with other nodes of the tree structure, starting with other nodes at a same level as the considered node, compares the computed pairwise similarity with a threshold to identify co-referring nodes that refer to a same entity, the threshold being a function of a depth of the compared node in the tree structure;
an information generator which outputs information based on the identified co-referring nodes, including enriching the database with the information on the identified co-referring nodes; and
a processor which implements the querying component, inference component, matching component, and information generator.
19. The system of claim 18 , further comprising a graphing component which generates an inferred ontology based on the tree structure and the identified co-referring objects.
20. A method for inferring coreference from serialized data comprising:
receiving responses from a plurality of different services, each of the responses comprising serialized data, the responses each identifying a hierarchical relationship between a respective set of objects represented by nodes of a sub-tree;
generating a single tree structure from the serialized data by connecting the highest level nodes of the sub-trees to a common root node, each of the nodes of the tree structure having a name corresponding to a respective one of the objects;
traversing the tree structure in a top-down, breadth first manner and for each considered node in the tree structure that is not a leaf node, starting with those of the other nodes in the tree structure at a same level of the tree structure:
computing a respective pairwise similarity with each of the other nodes of the tree structure, the computing of the pairwise similarity comprising computing a first similarity based on a similarity of the two nodes being compared and computing a second similarity based on a similarity of children of the two nodes being compared and aggregating the first and second similarities, and
comparing the computed pairwise similarity with a threshold to identify co-referring nodes; and
generating an inferred ontology in which children of co-referring nodes are linked to each of the co-referring nodes;
wherein at least one of the generating of the tree structure, computing a respective pairwise similarity, and identifying co-referring objects and generating an inferred ontology is performed with a processor.