Corpus clustering, confidence refinement, and ranking for geographic text search and information retrieval
View Patent ↗A computer-implemented method for processing a plurality of toponyms, the method involving: in a large corpus, identifying geo-textual correlations among readings of the toponyms within the plurality of toponyms; and for each toponym selected from the plurality of toponyms, using the identified geo-textual correlations to generate a value for a confidence that the selected toponym refers to a corresponding geographic location. Also a method of generating information useful for ranking a document that includes a plurality of toponyms for which there is a corresponding plurality of (toponym,place) pairs, there being associated with each (toponym,place) pair of said plurality of (toponym,place) pairs a corresponding value for a confidence that the toponym of that (toponym,place) pair refers to the place of that (toponym,place) pair. This further method includes, for a selected (toponym,place) pair of the plurality of (toponym,place) pairs, (1) determining if another toponym is present within the document that has an associated place that is geographically related to the place of the selected (toponym, place) pair; and (2) if a toponym is identified within the document that has an associated place that is geographically related to the place of the selected (toponym, place) pair, boosting the value of the confidence for the selected (toponym,place) pair.
1. A method comprising:
selecting, by a processor, a toponym-place pair of a target document that includes a plurality of toponyms corresponding to a plurality of toponym-place pairs, wherein the place of each toponym-place pair identifies a geographical location or region designated by the corresponding toponym; and
for the selected toponym-place pair, the method further comprises:
generating, by the processor, a confidence value, wherein the confidence value presents a confidence that the toponym of the selected toponym-place pair refers to the place of the selected toponym-place pair, and wherein the confidence value is pre-computed and derived from a statistical observation about a plurality of documents;
determining if another toponym is present within the target document that has an associated place that is geographically related to the place referred to by the selected toponym-place pair; and
if the another toponym is present within the target document that has an associated place that is geographically related to the place referred to by the selected toponym-place pair, boosting the generated confidence value for the selected toponym-place pair for the target document; and
ranking the target document based on the boosted confidence value.
2. The method of claim 1 , wherein determining if the another toponym is present within the target document that has an associated place that is geographically related to the place referred to by the selected toponym-place pair involves identifying the another toponym based, at least in part, on the another toponym having an associated geographic region that encompasses the place referred to by the selected toponym-place pair.
3. The method of claim 1 , wherein determining if the another toponym is present within the target document that has an associated place that is geographically related to the place referred to by the selected toponym-place pair includes identifying the another toponym based, at least in part, on the another toponym having an associated place that is geographically nearby the place referred to by the selected toponym-place pair.
4. The method of claim 3 , further comprising:
computing a geographical distance between the place associated with the identified another toponym and the place referred to by the selected toponym-place pair.
5. The method of claim 4 , wherein boosting involves calculating an adjustment value by computing an adjustment boosting function with the computed geographical distance as an input variable, said adjustment boosting function being monotonically decreasing for increasing values of the input variable.
6. The method of claim 5 , wherein boosting further involves deriving an initial boosting value from input including the calculated adjustment value.
7. The method of claim 6 , wherein boosting further involves applying a sigmoid function to the derived initial boosting value to compute a final boosting value and modifying the value of the confidence for the selected toponym-place pair by an amount determined by the final boosting value.
8. The method of claim 1 , wherein the associated place is different from the place referred to by the selected toponym-place pair.
9. An apparatus comprising:
at least one processor; and
at least one memory including computer program code for one or more programs, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,
select a toponym-place pair of a target document that includes a plurality of toponyms corresponding to a plurality of toponym-place pairs, wherein the place of each toponym-place pair identifies a geographical location or region designated by the corresponding toponym; and
for the selected toponym-place pair, the apparatus is further caused to:
generate a confidence value, wherein the confidence value presents a confidence that the toponym of the selected toponym-place pair refers to the place of the selected toponym-place pair, and wherein the confidence value is pre-computed and derived from a statistical observation about a plurality of documents;
determine if another toponym is present within the target document that has an associated place that is geographically related to the place referred to by the selected toponym-place pair; and
if the another toponym is present within the target document that has an associated place that is geographically related to the place referred to by the selected toponym-place pair, boosting the generated confidence value for the selected toponym-place pair for the target document; and
ranking the target document based on the boosted confidence value.
10. The apparatus of claim 9 , wherein determining if the another toponym is present within the target document that has an associated place that is geographically related to the place referred to by the selected toponym-place pair involves identifying the another toponym based, at least in part, on the another toponym having an associated geographic region that encompasses the place referred to by the selected toponym-place pair.
11. The apparatus of claim 9 , wherein determining if the another toponym is present within the target document that has an associated place that is geographically related to the place referred to by the selected toponym-place pair includes identifying the another toponym based, at least in part, on the another toponym having an associated place that is geographically nearby the place referred to by the selected toponym-place pair.
12. The apparatus of claim 11 , wherein the apparatus is further caused to:
compute a geographical distance between the place associated with the identified another toponym and the place referred to by the selected toponym-place pair.
13. The apparatus of claim 12 , wherein boosting involves calculating an adjustment value by computing an adjustment boosting function with the computed geographical distance as an input variable, said adjustment boosting function being monotonically decreasing for increasing values of the input variable.
14. The apparatus of claim 13 , wherein boosting further involves deriving an initial boosting value from input including the calculated adjustment value.
15. The apparatus of claim 14 , wherein boosting further involves applying a sigmoid function to the derived initial boosting value to compute a final boosting value and modifying the value of the confidence for the selected toponym-place pair by an amount determined by the final boosting value.
16. The apparatus of claim 9 , wherein the associated place is different from the place referred to by the selected toponym-place pair.
17. A non-transitory computer-readable storage medium carrying one or more sequences of one or more instructions for execution by one or more processors, wherein the executed one or more sequences of one or more instructions cause an apparatus to at least perform the following steps:
selecting a toponym-place pair of a target document that includes a plurality of toponyms corresponding to a plurality of toponym-place pairs, wherein the place of each toponym-place pair identifies a geographical location or region designated by the corresponding toponym; and
for the selected toponym-place pair the apparatus is caused to further perform:
generating a confidence value, wherein the confidence value presents a confidence that the toponym of the selected toponym-place pair refers to the place of the selected toponym-place pair, and wherein the confidence value is pre-computed and derived from a statistical observation about a plurality of documents;
determining if another toponym is present within the target document that has an associated place that is geographically related to the place referred to by the selected toponym-place pair; and
if the another toponym is present within the target document that has an associated place that is geographically related to the place referred to by the selected toponym-place pair, boosting the generated confidence value for the selected toponym-place pair for the target document; and
ranking the target document based on the boosted confidence value.
18. The non-transitory computer-readable storage medium of claim 17 , wherein determining if the another toponym is present within the target document that has an associated place that is geographically related to the place referred to by the selected toponym-place pair involves identifying the another toponym based, at least in part, on the another toponym having an associated geographic region that encompasses the place referred to by the selected toponym-place pair.
19. The non-transitory computer-readable storage medium of claim 17 , wherein determining if the another toponym is present within the target document that has an associated place that is geographically related to the place referred to by the selected toponym-place pair includes identifying the another toponym based, at least in part, on the another toponym having an associated place that is geographically nearby the place referred to by the selected toponym-place pair.
20. The non-transitory computer-readable storage medium of claim 19 , wherein the apparatus is caused to further perform:
computing a geographical distance between the place associated with the identified another toponym and the place referred to by the selected toponym-place pair.
21. The non-transitory computer-readable storage medium of claim 20 , wherein boosting involves calculating an adjustment value by computing an adjustment boosting function with the computed geographical distance as an input variable, said adjustment boosting function being monotonically decreasing for increasing values of the input variable.
22. The non-transitory computer-readable storage medium of claim 21 , wherein boosting further involves deriving an initial boosting value from input including the calculated adjustment value.
23. The non-transitory computer-readable storage medium of claim 22 , wherein boosting further involves applying a sigmoid function to the derived initial boosting value to compute a final boosting value and modifying the value of the confidence for the selected toponym-place pair by an amount determined by the final boosting value.
24. The non-transitory computer-readable storage medium of claim 17 , wherein the associated place is different from the place referred to by the selected toponym-place pair.