Systems and methods for scalable hierarchical coreference
A scalable hierarchical coreference method that employs a homomorphic compression scheme that supports addition and partial subtraction to more efficiently represent the data and the evolving intermediate results of probabilistic inference. The method may encode the features underlying conditional random field models of coreference resolution so that cosine similarities can be efficiently computed. The method may be applied to compressing features and intermediate inference results for conditional random fields. The method may allow compressed representations to be added and subtracted in a way that preserves the cosine similarities.
1. A system, comprising:
one or more storage devices;
one or more processors; and
a memory comprising program instructions executable by the one or more processors to:
store compressed entity information for a plurality of entities in a hierarchical conditional random field model on the one or more storage devices, wherein each entity is represented as an entity node that forms a subtree in the model, wherein mentions associated with each entity are stored as leaf nodes of the respective entity node;
wherein each mention is represented in the model as a low-dimensional fixed-width feature vector comprising n features, wherein the feature vectors are compressed according to a locality sensitive hash (LSH) function H that, for a feature vector a, stores a dot product a·h i as the hash of a such that H(a) is an array of dot products of length n.
2. The system as recited in claim 1 , wherein the program instructions are further executable by the one or more processors to compute cosine similarity between entity nodes in the model based on the compressed feature vectors.
3. The system as recited in claim 1 , wherein the program instructions are further executable by the one or more processors to add compressed feature vectors to the compressed entity information without requiring the feature vectors to be added to original uncompressed entity information and recompressing the entity information.
4. The system as recited in claim 1 , wherein the program instructions are further executable by the one or more processors to subtract compressed feature vectors from the compressed entity information without requiring the feature vectors to be subtracted from original uncompressed entity information and recompressing the entity information.
5. The system as recited in claim 1 , wherein the program instructions are further executable by at least one of the one or more processors to:
determine, based on the compressed entity information, that two entity nodes in the model refer to a same entity; and
merge the two entity nodes in the model without requiring the compressed entity information to be decompressed.
6. The system as recited in claim 5 , wherein, to merge the two entity nodes, the program instructions are further executable by the one or more processors to add the compressed feature vectors of the two entity nodes without requiring the compressed entity information to be decompressed.
7. The system as recited in claim 1 , wherein a subtree formed by an entity node has one or more intermediate nodes that organize subsets of the entity's mentions as leaf nodes, wherein the program instructions are further executable by at least one of the one or more processors to:
determine, based on the compressed entity information, that an intermediate node of an entity node refers to a different entity; and
split the intermediate node from the entity node's subtree in the model to form a new subtree in the model with the intermediate node as the entity node without requiring the compressed entity information to be decompressed.
8. The system as recited in claim 7 , wherein, to split the intermediate node from the entity node's subtree in the model, the program instructions are further executable by the one or more processors to subtract the compressed feature vectors of the intermediate node from the entity node without requiring the compressed entity information to be decompressed.
9. A method, comprising:
performing, by a computer system comprising one or more processors:
compressing feature vectors representing mentions associated with entities according to a locality sensitive hash (LSH) function H that, for a feature vector a, stores a dot product a·h i as the hash of a such that H(a) is an array of dot products of length n; and
storing the compressed feature vectors as compressed entity information for a plurality of entities in a hierarchical conditional random field model on one or more storage devices, wherein each entity is represented as an entity node that forms a subtree in the model, wherein compressed feature vectors representing mentions associated with each entity are stored as leaf nodes of the respective entity node.
10. The method as recited in claim 9 , further comprising computing cosine similarity between entity nodes in the model based on the compressed feature vectors without decompressing the entity information.
11. The method as recited in claim 9 , further comprising adding a compressed feature vector to the compressed entity information without decompressing the entity information.
12. The method as recited in claim 9 , further comprising subtracting a compressed feature vector from the compressed entity information without decompressing the entity information.
13. The method as recited in claim 9 , further comprising:
determining, based on the compressed entity information, that two entity nodes in the model refer to a same entity; and
merging the two entity nodes in the model without requiring the compressed entity information to be decompressed.
14. The method as recited in claim 13 , wherein merging the two entity nodes comprises adding the compressed feature vectors of the two entity nodes without requiring the compressed entity information to be decompressed.
15. The method as recited in claim 9 , wherein a subtree formed by an entity node has one or more intermediate nodes that organize subsets of the entity's mentions as leaf nodes, the method further comprising:
determining, based on the compressed entity information, that an intermediate node of an entity node refers to a different entity; and
splitting the intermediate node from the entity node's subtree in the model to form a new subtree in the model with the intermediate node as the entity node without requiring the compressed entity information to be decompressed.
16. The method as recited in claim 15 , wherein splitting the intermediate node from the entity node's subtree in the model comprises subtracting the compressed feature vectors of the intermediate node from the entity node without requiring the compressed entity information to be decompressed.
17. A non-transitory, computer-readable storage medium storing program instructions that when executed on one or more computers cause the one or more computers to:
compress feature vectors representing mentions associated with entities according to a locality sensitive hash (LSH) function H that, for a feature vector a, stores a dot product a·h i as the hash of a such that H(a) is an array of dot products of length n; and
store the compressed feature vectors as compressed entity information for a plurality of entities in a hierarchical conditional random field model on one or more storage devices, wherein each entity is represented as an entity node that forms a subtree in the model, wherein compressed feature vectors representing mentions associated with each entity are stored as leaf nodes of the respective entity node.
18. The non-transitory, computer-readable storage medium of claim 17 , wherein the program instructions when executed further cause the one or more computers to compute cosine similarity between entity nodes in the model based on the compressed feature vectors.
19. The non-transitory, computer-readable storage medium of claim 17 , wherein the program instructions when executed further cause the one or more computers to add a compressed feature vector to the compressed entity information without requiring the feature vector to be added to original uncompressed entity information and recompressing the entity information.
20. The non-transitory, computer-readable storage medium of claim 17 , wherein the program instructions when executed further cause the one or more computers to subtract a compressed feature vector from the compressed entity information without requiring the feature vector to be subtracted from original uncompressed entity information and recompressing the entity information.