Semantic hashing in entity resolution
According to one aspect, a computer-implemented method for entity resolution is disclosed. In one embodiment, the method includes generating a semantic hash for an entity having an assigned entity identifier (ID) and, upon the occurrence of an entity milestone, changing the entity ID. The method further includes generating a semantic hash for the entity having the changed entity ID, and maintaining history information associated with the entity and corresponding entity IDs and semantic hashes over a period of time that includes a plurality of entity milestones. The method also includes periodically removing at least one set of older entities and retaining entity IDs and semantic hashes associated with the removed entities.
1. A computer-implemented method comprising:
computing a semantic hash for an entity having an entity identifier (ID);
upon the occurrence of an entity milestone, changing the entity ID;
computing a semantic hash for the entity having the changed entity ID;
maintaining history information comprising entity IDs and semantic hashes associated with the entity over a period of time that includes a plurality of entity milestones; and
periodically removing at least one set of older entities and retaining entity IDs and semantic hashes associated with the removed entities.
2. The computer-implemented method of claim 1 , wherein an event milestone comprises a merge of a plurality of entities into a single entity corresponding to the same real-world individual.
3. The computer-implemented method of claim 1 , wherein an event milestone comprises a split of an entity into a plurality of entities corresponding to multiple real-world individuals.
4. The computer-implemented method of claim 1 , wherein an event milestone comprises removal of entities from a coreference set.
5. The computer-implemented method of claim 1 , wherein each of the semantic hashes comprises high confidence information associated with attributes of the entity.
6. The computer-implemented method of claim 5 , wherein the high confidence information comprises at least one of biographical information, distribution information associated with mention text, co-location information, assertion information, and temporal information.
7. The computer-implemented method of claim 1 , further comprising maintaining a search structure for performing an entity similarity search across the semantic hashes.
8. The computer-implemented method of claim 7 , wherein the search structure is configured to enable a nearest neighbor search to be performed across semantic hashes associated with the entity IDs.
9. A computing system comprising:
a processing unit;
a memory operatively coupled to the processing unit; and
a program module which executes in the processing unit from the memory and which, when executed by the processing unit, causes a computer to perform functions comprising:
computing a semantic hash for at least one entity having a respective at least one entity identifier (ID);
upon the occurrence of an entity milestone, changing the at least one entity ID;
computing a semantic hash for the at least one entity having the changed at least one entity ID;
maintaining history information comprising entity IDs and semantic hashes associated with the at least one entity over a period of time that includes a plurality of entity milestones; and
periodically removing at least one set of older entities and retaining entity IDs and semantic hashes associated with the removed entities.
10. The computing system of claim 9 , wherein an event milestone comprises at least one of:
a merge of a plurality of entities into a single entity corresponding to the same real-world individual;
a split of an entity into a plurality of entities corresponding to multiple real-world individuals; and
removal of entities from a coreference set.
11. The computing system of claim 9 , wherein each of the semantic hashes comprises high confidence information associated with attributes of the at least one entity.
12. The computing system of claim 10 , wherein the high confidence information comprises at least one of biographical information, distribution information associated with mention text, co-location information, assertion information, and temporal information.
13. The computing system of claim 9 , wherein the functions further comprise maintaining a search structure for performing an entity similarity search across the semantic hashes.
14. The computing system of claim 13 , wherein the search structure is configured to enable a nearest neighbor search to be performed across semantic hashes associated with the entity IDs.
15. A computer-readable storage medium having computer-executable instructions stored thereon which, when executed by a computer, cause the computer to perform functions comprising:
computing a semantic hash for at least one entity having a respective at least one entity identifier (ID);
upon the occurrence of an entity milestone, changing the at least one entity ID;
computing a semantic hash for the at least one entity having the changed at least one entity ID;
maintaining history information comprising entity IDs and semantic hashes associated with the at least one entity over a period of time that includes a plurality of entity milestones; and
periodically removing at least one set of older entities and retaining entity IDs and semantic hashes associated with the removed entities.
16. The computer-readable storage medium of claim 15 , wherein an event milestone comprises at least one of:
a merge of a plurality of entities into a single entity corresponding to the same real-world individual;
a split of an entity into a plurality of entities corresponding to multiple real-world individuals; and
removal of entities from a coreference set.
17. The computer-readable storage medium of claim 15 , wherein each of the semantic hashes comprises high confidence information associated with attributes of the at least one entity.
18. The computer-readable storage medium of claim 17 , wherein the high confidence information comprises at least one of biographical information, distribution information associated with mention text, co-location information, assertion information, and temporal information.
19. The computer-readable storage medium of claim 15 , wherein the functions further comprise maintaining a search structure for performing an entity similarity search across the semantic hashes.
20. The computer-readable storage medium of claim 19 , wherein the search structure is configured to enable a nearest neighbor search to be performed across semantic hashes associated with the entity IDs.