Semantic unit recognition
View Patent ↗A semantic locator determines whether input sequences form semantically meaningful units. The semantic locator includes a coherence component that calculates a coherence of the terms in the sequence and a variation component that calculates the variation in terms that surround the sequence. A heuristics component may additionally refine results of the coherence component and the variation component. A decision component may make the determination of whether the sequence is a semantic unit based on the results of the coherence component, variation component, and heuristics component.
1. A computer-implemented method of identifying named entities, the method comprising:
receiving, by one or more processors of the computer, an electronic mail document or a web page document that includes a sequence of terms;
calculating, by the one or more processors, a first value representing a coherence of the terms included the sequence;
calculating, by the one or more processors, a second value representing an entropy of contexts associated with the sequence;
determining, by the one or more processors, that the sequence is associated with a named entity based on:
the first value satisfying a first threshold, and
the second value satisfying a second threshold; and
storing, by the one or more processors, an indication that the sequence is associated with the named entity.
2. The computer-implemented method of claim 1 , further comprising:
receiving a search query that includes the sequence of terms; and
presenting, for display, search results associated with the named entity.
3. The computer-implemented method of claim 1 , where the named entity refers to a person.
4. The computer-implemented method of claim 1 , where the named entity refers to a place.
5. The computer-implemented method of claim 1 , where the named entity refers to a company.
6. The computer-implemented method of claim 1 , where calculating the first value further includes:
identifying a first quantity associated with occurrences, in a collection of documents, of the sequence;
identifying second quantities associated with respective occurrences, in the collection of documents, of the terms included in the sequence; and
determining the first value based on the first quantity and the second quantities.
7. The computer-implemented method of claim 1 , where calculating the second value further includes:
identifying occurrences, in a collection of documents, of the sequence;
identifying, based on the occurrences, additional terms that appear adjacent to the sequence in the collection of documents;
determining respective quantities associated with appearances of the additional terms in the collection of documents; and
calculating the second value based on the respective quantities.
8. A system identifying named entities, the system comprising:
a computing device to:
receive an electronic mail document or a web page document that includes a sequence of terms;
calculate a first value representing a coherence of the terms included the sequence;
calculate a second value representing an entropy of contexts associated with the sequence;
determine that the sequence is associated with a named entity based on:
the first value satisfying a first threshold, and
the second value satisfying a second threshold; and
store an indication that the sequence is associated with the named entity.
9. The system of claim 8 , where the computing device, when determining that the sequence is associated with the named entity, is further to:
apply one or more heuristic rules to the sequence to form a modified sequence,
the first value and the second value being calculated based on the modified sequence.
10. The system of claim 8 , where the named entity refers to a person.
11. The system of claim 8 , where the named entity refers to a place.
12. The system of claim 8 , where the named entity refers to a company.
13. The system of claim 8 , where the computing device, when calculating the first value, is further to:
identify a first quantity associated with occurrences, in a collection of documents, of the sequence;
identify second quantities associated with respective occurrences, in the collection of documents, of the terms included in the sequence; and
determine the first value based on the first quantity and the second quantities.
14. The system of claim 8 , where the computing device, when calculating the second value, is further to:
identify occurrences, in a collection of documents, of the sequence;
identify, based on the occurrences, additional terms that appear adjacent to the sequence in the collection of documents;
determine respective quantities associated with appearances of the additional terms in the collection of documents; and
calculate the second value based on the respective quantities.
15. A non-transitory computer-readable medium to store instructions to identify named entities, the instructions comprising:
one or more instructions which, when executed by one or more processors associated with a computing device, cause the one or more processors to:
receive an electronic mail document or a web page document that includes a sequence of terms;
calculate a first value representing a coherence of the terms included the sequence;
calculate a second value representing an entropy of contexts associated with the sequence;
determine that the sequence is associated with a named entity based on:
the first value satisfying a first threshold, and
the second value satisfying a second threshold; and
store an indication that the sequence is associated with the named entity.
16. The computer-readable medium of claim 15 , where the named entity refers to a person.
17. The computer-readable medium of claim 15 , where the named entity refers to a place.
18. The computer-readable medium of claim 15 , where the named entity refers to a company.
19. The non-transitory computer-readable medium of claim 15 , where the one or more instructions to calculate the first value further include:
one or more instructions to:
identify a first quantity associated with occurrences, in a collection of documents, of the sequence;
identify second quantities associated with respective occurrences, in the collection of documents, of the terms included in the sequence; and
determine the first value based on the first quantity and the second quantities.
20. The computer-readable medium of claim 15 , where the one or more instructions to calculate the second value further include:
one or more instructions to:
identify occurrences, in a collection of documents, of the sequence;
identify, based on the occurrences, additional terms that appear adjacent to the sequence in the collection of documents;
determine respective quantities associated with appearances of the additional terms in the collection of documents; and
calculate the second value based on the respective quantities.