Cloaking detection utilizing popularity and market value
View Patent ↗The subject disclosure pertains to systems and methods that facilitate detection of cloaked web pages. Commercial value of search terms and/or queries can be indicative of the likelihood that web pages associated with the keywords or queries are cloaked. Commercial value can be determined based upon popularity of terms and/or advertisement market value as established based upon advertising revenue, fees and the like. Commercial value can be utilized in conjunction with term frequency difference analysis to identify a cloaked page automatically. In addition, commercial values of terms associated with web pages can be used to order or prioritize web pages for further analysis.
1. A computer-implemented system that facilitates detection of a cloaked web page, comprising:
a memory having computer executable components stored thereon; and
a processor communicatively coupled to the memory, the processor configured to execute the computer executable components, the computer executable components comprising:
a term analysis component configured to determine a term score for at least one term included in a content of a web page, wherein the term score is based on a commercial value assigned to the at least one term; and
a page analysis component configured to compute a page score based on a difference in commercial values between a first set of terms in a browser version of the web page and a second set of terms in a web crawler version of the web page, wherein the page score is indicative of a probability that the web page is cloaked.
2. The system of claim 1 , further comprising a popularity component configured to evaluate a popularity of the at least one term, wherein the term score is based at least in part upon the popularity.
3. The system of claim 2 , the popularity is based at least in part upon usage data obtained from a search query log.
4. The system of claim 1 , further comprising a market value component configured to evaluate a market value of the at least one term, wherein the term score is based at least in part upon the market value.
5. The system of claim 4 , wherein the market value is based at least in part upon web advertisement monetization data.
6. The system of claim 1 , further comprising a page order component configured to prioritize a set of web pages as a function of the term score, wherein the web page is a member of the set of web pages.
7. The system of claim 1 , wherein the difference in commercial values includes a normalized term frequency difference.
8. A method that facilitates detection of web spam, comprising:
employing a processor to execute computer executable instructions stored on a non-transitory computer readable storage medium to implement the following acts:
identifying at least one term embedded within a content of a web page;
determining a term score for the at least one term, wherein the term score is of based on a commercial value assigned to the at least one term;
computing a difference in commercial values between a first set of terms in a browser version of the web page and a second set of terms in a web crawler version of the web page; and
ascertaining a probability that the web page is a web spam as a function of the difference in commercial values.
9. The method of claim 8 , further comprising:
determining a popularity of the at least one term; and
determining a market value of the at least one term, wherein the term score is a function of at least one of the popularity or the market value of the at least one term.
10. The method of claim 9 , wherein the term score is a function of a frequency of occurrence of the at least one term.
11. The method of claim 9 , wherein the term score is a function of advertising monetization information associated with the at least one term.
12. The method of claim 8 , further comprising:
performing a comparison between the term score and at least one threshold value; and
classifying the web page as a function of the comparison.
13. The method of claim 8 , wherein the at least one term is identified based at least in part upon query log information associated with the web page.
14. The method of claim 8 , wherein the at least one string term is included in a set of terms that differ between the crawler version and the browser version of the web page.
15. The method of claim 8 , further comprising ordering a set of web pages based at least in part upon the term score, wherein the web page is a member of the set of web pages.
16. The method of claim 8 , further comprising:
generating a cloaking score as a function of the difference in commercial values.
17. The method of claim 8 , further comprising determining a normalized term frequency difference between the browser version and the crawler version of the web page.
18. The method of claim 8 , wherein the term score is based at least in part upon a position of the at least one term on the web page.
19. A computer-implemented system that facilitates detection of a cloaked web page, comprising:
a memory having computer executable components stored thereon; and
a processor communicatively coupled to the memory, the processor configured to execute the computer executable components, the computer executable components comprising:
a popularity component configured to determine a popularity of at least one term included in a content of a web page as a function of a frequency of occurrence of the at least one term;
a market value component configured to evaluate a market value of the at least one term as a function of advertisement monetization information;
a term analysis component configured to determine a term score based on a commercial value assigned to the at least one term, wherein the commercial value is a function of the popularity and the market value; and
a page analysis component configured to facilitate an evaluation of a probability that the web page is cloaked based at least in part upon a difference in commercial values between a first set of terms in a browser version of the web page and a second set of terms in a web crawler version of the web page.