Method and system for entity linking
The present teaching relates to entity linking. In one example, a text string is received. The text string is segmented to obtain a segmentation with a set of one or more segments of the text string. A set of entities are identified, with respect to the one or more segments, from a plurality of entities as linked to the one or more segments. The identifying is in accordance with a probabilistic model based on surface form information associated with the plurality of entities.
1. A method, implemented on a computing device having at least one processor, storage, and a communication platform capable of connecting to a network for entity linking, comprising:
receiving a search query, including a first text string, from a user;
segmenting the first text string to obtain a plurality of segments, wherein each of the plurality of segments is a portion of the first text string;
associating a first set of entities from a plurality of entities with one of the plurality of segments of the first text string based on a probabilistic model, the probabilistic model being based at least partially on a previously submitted search query, which includes a second text string and which led the user to Internet content affiliated with at least one entity of the first set of entities, wherein the probabilistic model is used to compute a probabilistic score for each of the plurality of entities with respect to the one segment of the first text string, and wherein the probabilistic scores are computed independently of one another; and
identifying the first set of entities from the plurality of entities as linked to the one segment of the first text string based on the association.
2. The method of claim 1 , wherein the probabilistic model is based on an anchor text representing a link to a web page associated with an entity.
3. The method of claim 1 , wherein the step of identifying comprises:
computing, for the one segment and with respect to each of the plurality of entities, the probabilistic score P(e|s) for the one segment based at least partially on the search query, wherein the probabilistic score P(e|s) is indicative of a likelihood that entity e is a linked entity given segment s.
4. The method of claim 3 , further comprising:
identifying a second set of entities from the plurality of entities as linked to another one segment of the plurality of segments, in accordance with the probabilistic model based at least partially on a previously submitted search query that led a user to Internet content associated with at least one entity of the second set of entities; and
determining a set of linked entities based on the first set of entities and the second set of entities.
5. The method of claim 4 , wherein the set of linked entities are determined by:
maximizing aggregated probabilistic scores associated with the one segment and the other one segment; or
maximizing one of the probabilistic scores associated with the one segment and the other one segment.
6. The method of claim 1 , wherein the identifying is based, at least in part, on context of the first text string.
7. The method of claim 6 , wherein
the probabilistic model includes a contextual relevance model based on which a context sensitive probabilistic score is computed for each entity; and
the context sensitive probabilistic score is indicative of a likelihood that the entity is linked to a segment given the context of the first text string.
8. The method of claim 7 , wherein the context sensitive probabilistic score is determined based on a similarity between a first vector representing content associated with the entity and a second vector representing the context of the first text string.
9. The method of claim 8 , wherein the first vector is determined based on the content of a web page associated with the entity.
10. The method of claim 1 , further comprising:
determining, based on the identified linked entities, content sources where content related to the first text string can be retrieved.
11. The method of claim 1 , wherein each entity of the first set of entities is a thing that has a distinct and independent existence.
12. The system of claim 1 , wherein the first set of entities are identified as linked to the one segment before retrieving content responsive to the received search query.
13. The method of claim 1 , wherein the segmenting of the first text string to obtain the plurality of segments comprises:
tokenizing the first text string to generate a series of tokens;
generating a plurality of disjoint segmentations based on the series of tokens; and
segmenting one of the plurality of disjoint segmentations to obtain the plurality of segments.
14. The method of claim 13 , wherein the tokenizing comprises removing punctuation from the first text string, and wherein each of the tokens is a single term or n-gram.
15. The method of claim 13 , wherein the generating of the plurality of disjoint segmentations comprises computation of a score for each of the segmentations based on a scoring function.
16. A method, implemented on a computing device having at least one processor, storage, and a communication platform capable of connecting to a network for providing search results, comprising:
receiving a search query, including a first text string, from a user;
partitioning the first text string into a plurality of segments, wherein each of the plurality of segments is a portion of the first text string;
associating a first set of entities from a plurality of entities with one of the plurality of segments of the first text string based on a probabilistic model, the probabilistic model being based at least partially on a previously submitted search query, which includes a second text string and which led the user to Internet content affiliated with at least one entity of the first set of entities, wherein the probabilistic model is used to compute a probabilistic score for each of the plurality of entities with respect to the one segment of the first text string, and wherein the probabilistic scores are computed independently of one another;
identifying the first set of entities from the plurality of entities as linked to the one segment of the first text string based on the association;
identifying content sources associated with the first set of entities linked to the one segment;
determining search results from the content sources based on the search query; and
providing the search results as a response to the search query.
17. A system for entity linking, comprising:
a segmenting module configured to segment a first text string received as part of a search query from a user to obtain a plurality of segments, wherein each of the plurality of segments is a portion of the first text string; and
an entity identifying module coupled with the segmenting module and configured to associate a first set of entities from a plurality of entities with one of the plurality of segments based on a probabilistic model, the probabilistic model being based at least partially on a previously submitted search query, which includes a second text string and which led the user to Internet content affiliated with at least one entity of the first set of entities, wherein the probabilistic model is used to compute a probabilistic score for each of the plurality of entities with respect to the one segment of the first text string, and wherein the probabilistic scores are computed independently of one another, the entity identifying module being further configured to identify the first set of entities from the plurality of entities as linked to the one segment of the first text string based on the association.
18. The system of claim 17 , wherein the probabilistic model is based on an anchor text representing a link to a web page associated with an entity.
19. The system of claim 17 , wherein the entity identifying module comprises:
a probabilistic score calculator configured to compute, for the one segment and with respect to each of the plurality of entities, the probabilistic score P(e|s) for the one segment based at least partially on the search query, wherein the probabilistic score P(e|s) is indicative of a likelihood that entity e is a linked entity given segment s.
20. The system of claim 19 , wherein
the entity identifying module is further configured to:
identify a second set of entities from the plurality of entities as linked to another one segment of the plurality of segments, in accordance with the probabilistic model based at least partially on a previously submitted search query that led a user to Internet content associated with at least one entity of the second set of entities, and
determine a set of linked entities based on the first set of entities and the second set of entities.
21. The system of claim 20 , wherein the entity identifying module comprises an entity selector configured to:
maximize aggregated probabilistic scores associated with the one segment and the other one segment; or
maximize one of the probabilistic scores associated with the one segment and the other one segment.
22. The system of claim 17 , wherein the identifying is based, at least in part, on context of the first text string.
23. The system of claim 22 , wherein
the probabilistic model includes a contextual relevance model based on which a context sensitive probabilistic score is computed for each entity; and
the context sensitive probabilistic score is indicative of a likelihood that the entity is linked to a segment given the context of the first_text string.
24. The system of claim 23 , wherein the context sensitive probabilistic score is determined based on a similarity between a first vector representing content associated with the entity and a second vector representing the context of the first text string.
25. The system of claim 24 , wherein the first vector is determined based on the content of a web page associated with the entity.
26. A machine-readable, non-transitory and tangible medium having data recorded thereon for entity linking, the medium, when read by the machine, causes the machine to perform the following steps:
receiving a first text string as part of a search query from a user;
segmenting the first text string to obtain a plurality of segments, wherein each of the plurality of segments is a portion of the first text string;
associating a first set of entities from a plurality of entities with one of the plurality of segments of the first text string based on a probabilistic model, the probabilistic model being based at least partially on a previously submitted search query, which includes a second text string and which led the user to Internet content affiliated with at least one entity of the first set of entities, wherein the probabilistic model is used to compute a probabilistic score for each of the plurality of entities with respect to the one segment of the first text string, and wherein the probabilistic scores are computed independently of one another; and
identifying the first set of entities from the plurality of entities as linked to the one segment of the first text string based on the association.
27. The medium of claim 26 , wherein the steps further comprise:
identifying a second set of entities from the plurality of entities as linked to another one segment of the plurality of segments, in accordance with the probabilistic model based at least partially on a previously submitted search query that led a user to Internet content associated with at least one entity of the second set of entities; and
determining a selected set of linked entities based on the first set of entities and the second set of entities.
28. The medium of claim 26 , wherein each entity of the first set of entities is a thing that has a distinct and independent existence.