Affinity scoring
Some embodiments provide a method for determining a relatedness of content items to categories. The method identifies a particular content item, a relevancy score associated with the particular content item, and a set of categories to which the particular content item is classified as related. Based on a set of glossaries associated with the set of categories, the method calculates a set of affinity scores that each represents a degree of relevancy between the particular content item and a category in the set of categories. The method modifies the relevancy score associated with the particular content item based on the calculated set of affinity scores.
1. A system comprising:
one or more processors; and
a memory storing code stored therein for determining a relatedness of content items to categories, wherein when executed the code causes the one or more processors to perform operations comprising:
pre-processing content to obtain information in the content, classify the content as pertaining to one or more categories based on the information, and organizing the content in terms of relevancy to categories;
identifying a particular content item of the pre-processed content, a relevancy score associated with the particular content item, and a set of categories to which the particular content item is classified as related;
generating glossaries associated with the set of categories, wherein generating each glossary of the glossaries comprises:
using a glossary manager:
directing a word stemming module to stem words in a collection of content to reduce words to a base form;
receiving the collection of content with the words stemmed;
identifying business content that is tagged to a particular category using a word frequency module and a glossary word score module to calculate glossary word scores for words that occur in the content tagged to the particular category; and
applying a frequency threshold when generating the glossary and excluding a particular word in the glossary when the particular word does not occur in at least a predetermined number of pieces of content that is tagged to the particular category:
based on probability values in the set of glossaries associated with the set of categories, calculating a set of affinity scores that each represent a degree of relevancy between the particular content item and a category in the set of categories, wherein each glossary associated with a particular category in the set of categories comprises a set of words and a corresponding set of glossary word scores that represent the probability values that a given content items are related to the particular category when the content items contain the word associated with the glossary word score; and
modifying the relevancy score associated with the particular content item based on the calculated set of affinity scores.
2. The system of claim 1 , wherein modifying the relevancy score associated with the particular content item comprises calculating a weighted sum of the set of affinity scores based on degrees to which the set of categories is classified as related to the particular content item.
3. The system of claim 1 , wherein modifying the relevancy score associated with the particular content item comprises normalizing the relevancy score associated with the particular content item and the set of affinity scores.
4. The system of claim 1 , wherein the set of categories is a set of industries.
5. The system of claim 1 , wherein the particular content item comprises a word that is identified as an entity.
6. The system of claim 1 , wherein the set of categories to which the particular content item is classified as related is based on a business web graph comprising a node that represents the entity and a set of nodes that represents the set of categories.
7. The system of claim 1 , wherein the code is further executable by the one or more processors to cause the processor to perform:
identifying a set of glossaries, each glossary associated with a category in the set of categories and comprising a set of (i) words and (ii) corresponding glossary word scores that each represents the probability that a given content item is related to the category when the content item contains the word associated with the glossary word score.
8. A system comprising:
one or more processors; and
a memory storing code stored therein for determining a relatedness of content items to categories, wherein when executed the code causes the one or more processors to perform operations comprising:
pre-processing content to obtain information in the content, classify the content as pertaining to one or more categories based on the information, and organizing the content in terms of relevancy to categories, wherein the pre-processing of the content comprises:
generating glossaries associated with the set of categories, wherein generating each glossary of the glossaries comprises:
using a glossary manager:
directing a word stemming module to stem words in a collection of content to reduce words to a base form;
receiving the collection of content with the words stemmed;
identifying business content that is tagged to a particular category using a word frequency module and a glossary word score module to calculate glossary word scores for words that occur in the content tagged to the particular category;
applying a frequency threshold when generating the glossary and excluding a particular word in the glossary when the particular word does not occur in at least a predetermined number of pieces of content that is tagged to the particular category;
providing an affinity scoring manager for identifying a particular content item of the pre-processed content including at least one of the glossaries, a relevancy score associated with the particular content item, and a set of categories to which the particular content item is classified as related;
providing a word score module for calculating a set of affinity scores that each represents a degree of relevancy between the particular content item and a category in the set of categories, wherein (i) the calculating is based on probability values in a set of glossaries associated with the set of categories and (ii) each glossary associated with a particular category in the set of categories comprises a set of words and a corresponding set of glossary word scores that represent the probability values that a given content items are related to the particular category when the content items contain the word associated with the glossary word score; and
providing a score calculator for calculating a weighted sum of the set of affinity scores based on degrees to which the set of categories is classified as related to the particular content item, wherein the affinity scoring manager is further for modifying the relevancy score associated with the particular content item based on the calculated set of affinity scores.
9. The system of claim 8 , wherein modifying the relevancy score associated with the particular content item comprises calculating a weighted sum of the set of affinity scores based on degrees to which the set of categories is classified as related to the particular content item.
10. The system of claim 8 , wherein modifying the relevancy score associated with the particular content item comprises normalizing the relevancy score associated with the particular content item and the set of affinity scores.
11. The system of claim 8 , wherein the set of categories is a set of industries.
12. The system of claim 8 , wherein the particular content item comprises a word that is identified as an entity.
13. The system of claim 8 , wherein the set of categories to which the particular content item is classified as related is based on a business web graph comprising a node that represents the entity and a set of nodes that represents the set of categories.
14. The system of claim 8 , wherein the code is further executable by the processor to cause the processor to perform:
identifying a set of glossaries, each glossary associated with a category in the set of categories and comprising a set of (i) words and (ii) corresponding glossary word scores that each represents the probability that a given content item is related to the category when the content item contains the word associated with the glossary word score.
15. The system of claim 8 , wherein the affinity scoring manager modifies the relevancy score by normalizing the relevancy score associated with the particular content item and the set of affinity scores.
16. The system of claim 8 , wherein the set of categories is a set of industries.
17. The system of claim 8 , wherein modifying the relevancy score associated with the particular content item comprises calculating a weighted sum of the set of affinity scores based on degrees to which the set of categories is classified as related to the particular content item.
18. The system of claim 8 , wherein modifying the relevancy score associated with the particular content item comprises normalizing the relevancy score associated with the particular content item and the set of affinity scores.