Detecting duplicates in a shared knowledge base
View Patent ↗Methods and systems supporting curation of items in a searchable knowledge base are provided. The methods and systems include mining one or more search queries of the searchable knowledge base, where each of the search queries includes a plurality of the items. The method further includes determining one or more pairs of items using a processor, where each of the pairs of items includes a correlation value exceeding a threshold. The correlation values for the pairs of items are based upon the frequency the items of the pairs of items co-occur within the search queries. The method further includes providing the pairs of items to a curator, where the curator reviews the pairs of items.
1. A method for supporting curation of items in a searchable knowledge base, said method comprising:
mining one or more search queries of the searchable knowledge base, wherein each of the one or more search queries includes a plurality of the items, each item including a title identifying an issue and/or topic addressed by the item and further including a plurality of posts from users of the knowledge base system, each post relevant to the issue and/or topic of the item;
determining one or more pairs of items using a processor, wherein each of the one or more pairs of items include a correlation value exceeding a threshold, wherein the correlation value for the each of the one or more pairs of items is based upon a frequency the items of the each of the one or more pairs of items co-occur within the one or more search queries; and
providing at least one of the one or more pairs of items to a curator, wherein the curator reviews the one or more pairs of items;
wherein the one or more mined search queries are re-run before determining the one or more pairs or items, and wherein the one or more mined search queries are saved within a history database, each of the saved search queries corresponding to a previously performed search of the searchable knowledge base, identifying the search terms of the previously performed search, and identifying items of the searchable knowledge base returned in response to the previously performed search.
2. The method of claim 1 , wherein the correlation value for the each of the one or more pairs of items is further based upon a frequency with which terms co-occur within the items of the each of the one or more pairs of items.
3. The method of claim 1 , wherein the items of the searchable knowledge base are search tips.
4. The method of claim 1 , wherein the curator removes duplicate items or expired items from the one or more pairs of items.
5. The method of claim 1 , further comprising:
ranking the each of the one or more pairs of items according the frequency the items of the each of the one or more pairs of items co-occur within the one or more search queries, wherein the each of the one or more pairs of items are ranked before the at least one of one or more pairs of items are provided to the curator.
6. The method of claim 5 , where the at least one of the one or more pairs of items are provided to the curator until the curator says otherwise.
7. The method according to claim 1 , wherein each of the one or more mined search queries includes search results including the plurality of the items.
8. The method according to claim 1 , wherein the items in the knowledge base each correspond to a record of information on a specific issue or topic relevant to a topic covered by the knowledge base.
9. A method for supporting curation of a new item in a searchable knowledge base, wherein the searchable knowledge base includes a plurality of items, said method comprising:
mining one or more search queries of the searchable knowledge base, wherein each of the one or more search queries includes search results including a plurality of the plurality of items of the knowledge base, each item including a title identifying an issue and/or topic addressed by the item and further including a plurality of posts from users of the knowledge base system, each post relevant to the issue and/or topic of the item;
re-running at least one of the one or more mined search queries against the searchable knowledge base using a keyword based search algorithm;
determining one or more items of the plurality of items of the knowledge base using a processor, wherein each of the one or more items include a correlation value exceeding a threshold, wherein the correlation value for the each of the one or more items is based upon a frequency the each of the one or more items co-occur with the new item in search results of at least one of the at least one of the one or more search queries; and
providing at least one of the one or more items to a curator, wherein the curator reviews the at least one of the one or more items;
wherein the plurality of items of the knowledge base each correspond to a record of information on a specific issue or topic relevant to a topic covered by the knowledge base, and wherein the one or more mined search queries are saved within a history database, each of the saved search queries corresponding to a previously performed search of the searchable knowledge base, identifying the search terms of the previously performed search, and identifying items of the searchable knowledge base returned in response to the previously performed search.
10. The method of claim 9 , further comprising:
determining which of the one or more search queries are likely to return the new item, wherein the at least one of the one or more search queries are the determined searched queries.
11. The method of claim 9 , further comprising:
ranking the each of the one or more items according the frequency the each of the one or more items co-occur with the new item, wherein the each of the one or more items are ranked before the at least one of one or more items are provided to the curator.
12. The method of claim 11 , where the at least one of the one or more items are provided to the curator until the curator says otherwise.
13. The method of claim 9 , further comprising:
temporarily augmenting the knowledge base to include the new item before re-running the at least one of the one or more mined search queries.
14. The method of claim 9 , further comprising:
determining which of the at least one of the one or more search queries include the new item ranked above a threshold, wherein the at least one of the at least one of the one or more search queries are the determined search queries.
15. The method of claim 9 , wherein the mined one or more search queries exclude the new item.
16. The method of claim 9 , wherein the correlation value for the each of the one or more items is further based upon a frequency with which terms of the each of the one or more items co-occur with terms of the new item.
17. The method of claim 16 , wherein the frequency with which terms of the each of the one or more items co-occur with terms of the new item includes an inverse document frequency factor.
18. The method of claim 9 , wherein the plurality of items of the searchable knowledge base are search tips.
19. The method of claim 9 , wherein the curator removes duplicate items or expired items from the at least one of the one or more items.
20. A system for supporting curation of items in a searchable knowledge base, said system comprising:
at least one processor configured to:
mine one or more search queries of the searchable knowledge base, wherein each of the one or more search queries includes search results including a plurality of the items of the knowledge base, each item including a title identifying an issue and/or topic addressed by the item and further including a plurality of posts from users of the knowledge base system, each post relevant to the issue and/or topic of the item;
determine one or more pairs of items of the items of the knowledge base using a processor, wherein each of the one or more pairs of items include a correlation value exceeding a threshold, wherein the correlation value for the each of the one or more pairs of items is based upon a frequency the items of the each of the one or more pairs of items co-occur within search results of the one or more search queries; and
provide at least one of the one or more pairs of items to a curator, wherein the curator reviews the one or more pairs of items;
wherein the items of the knowledge base each correspond to a record of information on a specific issue or topic relevant to a topic covered by the knowledge base;
wherein the one or more mined search queries are re-run before determining the one or more pairs or items, and wherein the one or more mined search queries are saved within a history database, each of the saved search queries corresponding to a previously performed search of the searchable knowledge base, identifying the search terms of the previously performed search, and identifying items of the searchable knowledge base returned in response to the previously performed search.
21. The system of claim 20 , wherein the correlation value for the each of the one or more pairs of items is further based upon a frequency with which terms co-occur within the items of the each of the one or more pairs of items.
22. The system of claim 20 , wherein the items of the searchable knowledge base are search tips.
23. The system of claim 20 , wherein the curator removes duplicate items or expired items from the one or more pairs of items.
24. A system for supporting curation of a new item in a searchable knowledge base, wherein the searchable knowledge base includes a plurality of items, said system comprising:
at least one processor configured to:
mine one or more search queries of the searchable knowledge base, wherein each of the one or more search queries includes search results including a plurality of the plurality of items of the knowledge base, each item including a title identifying an issue and/or topic addressed by the item and further including a plurality of posts from users of the knowledge base system, each post relevant to the issue and/or topic of the item;
re-run at least one of the one or more mined search queries against the searchable knowledge base using a PageRank algorithm, the PageRank algorithm using information external to the searchable knowledge base;
determine one or more items of the plurality of items of the knowledge base using a processor, wherein each of the one or more items include a correlation value exceeding a threshold, wherein the correlation value for the each of the one or more items is based upon a frequency the each of the one or more items co-occur with the new item in search results of at least one of the at least one of the one or more search queries; and
provide at least one of the one or more items to a curator, wherein the curator reviews the at least one of the one or more items;
wherein the plurality of items of the knowledge base each correspond to a record of information on a specific issue or topic relevant to a topic covered by the knowledge base, and wherein the one or more mined search queries are saved within a history database, each of the saved search queries corresponding to a previously performed search of the searchable knowledge base, identifying the search terms of the previously performed search, and identifying items of the searchable knowledge base returned in response to the previously performed search.
25. The system of claim 24 , wherein the mined one or more search queries exclude the new item.
26. The system of claim 24 , wherein the correlation value for the each of the one or more items is further based upon a frequency with which terms of the each of the one or more items co-occur with terms of the new item.
27. The system of claim 24 , wherein the items of the searchable knowledge base are search tips.
28. The system of claim 24 , wherein the curator removes duplicate items or expired items from the at least one of the one or more items.