System and method for analyzing web content
A system and computer based method are provided for identifying active content in websites on a network. One embodiment includes a computer based method of classifying web content. The method receives content of a web page, and determines a first property associated with the content, the first property including static content. The method executes active content associated with the webpage, and determines a second property associated with the content based at least in part on the executing, the second property including the active content. The method also evaluates a logical expression relating the first property and the second property, and associates the web page with a category based on a result of the evaluation. The evaluation of the logical expression at least in part evaluates whether a constant value matches at least a portion of the content of the web page.
1. A method of classifying web content, implemented on one or more computer processors, the method comprising:
using at least one of the processors, receiving content of a web page;
using at least one of the processors, determining a first property associated with the content of the web page, the first property comprising static content associated with the web page;
using at least one of the processors, executing active content associated with the webpage;
using at least one of the processors, determining a second property associated with the content of the web page based at least in part on the executing, the second property comprising the active content;
using at least one of the processors, evaluating a logical expression relating the first property and the second property; and
using at least one of the processors, associating the web page with a category based on a result of the evaluation,
wherein the evaluation of the logical expression at least in part evaluates whether a constant value matches at least a portion of the content of the web page.
2. The method of claim 1 , wherein executing the active content comprises executing the active content in a sandbox environment.
3. The method of claim 1 , wherein determining the first property associated with content of the web page comprises identifying a keyword in the content of the web page.
4. The method of claim 1 , wherein the first property is associated with a regular expression matching at least a portion of the content of the web page.
5. The method of claim 1 , wherein the first property includes an indication of a number of occurrences of a string within the web page.
6. The method of claim 1 , wherein evaluating the logical expression comprises relating each of the first and second properties to corresponding constant values, and wherein the method further comprises assigning the category to the web page if the evaluated logical expression corresponds to a predetermined value.
7. The method of claim 6 , wherein relating the first property to its corresponding constant value comprises evaluating a regular expression based on a content of the web page, and relating the result of the evaluation to the corresponding constant value.
8. The method of claim 1 , further comprising:
evaluating a plurality of logical expressions based on the content of the web page, wherein each of the logical expressions relate the first property and the second property, wherein each of the plurality of logical expressions corresponds to a web page category; and
associating the web page with a logical expression's corresponding category if the evaluated logical expression corresponds to a predetermined value.
9. A system for classifying web content, the system comprising:
one or more hardware processors configured, individually or in combination, to:
receive content of a web page;
determine a first property associated with the content of the web page, the first property comprising static content associated with the web page;
determine a second property associated with the content of web page based at least in part by executing active content associated with the webpage, the second property comprising the active content;
evaluate a logical expression relating the first property and the second property; and
associate the web page with a category based on a result of the evaluation,
wherein the evaluation at least in part evaluates whether a constant value matches at least a portion of the content of the web page.
10. The system of claim 9 , wherein the one or more processors are further configured, individually or in combination, to execute the active content in a sandbox environment.
11. The system of claim 9 , wherein the one or more processors are further configured, individually or in combination, to determine the first property of the web page at least in part by identifying a keyword in the content of the web page.
12. The system of claim 9 , wherein the first property is associated with a regular expression matching at least a portion of the content of the web page.
13. The system of claim 9 , wherein the first property includes an indication of a number of occurrences of a string within the web page.
14. The system of claim 9 , wherein evaluating the logical expression comprises relating each of the first and second properties to corresponding constant values, and wherein the one or more processor are further configured to assign the category to the web page if the evaluated logical expression corresponds to a predetermined value.
15. The system of claim 14 , wherein relating the first property to its corresponding constant value comprises evaluating a regular expression on a content of the web page, and relating the result of the evaluation to the corresponding constant value.
16. The system of claim 9 , wherein the one or more processors are further configured, individually or in combination, to:
evaluate a plurality of logical expressions based on the content of the web page, wherein each of the logical expressions relates the first property and the second property, and wherein each of the plurality of logical expressions corresponds to a web page category; and
associate the web page with a logical expression's corresponding category if the evaluated logical expression corresponds to a predetermined value.
17. An apparatus for classifying web content, implemented on one or more computer processors, comprising:
means for determining a first property associated with content of a web page, the first property comprising static content associated with the web page;
means for executing active content associated with the webpage;
means for determining a second property associated with the content of the web page based at least in part on the executing, the second property comprising the active content;
means for evaluating a logical expression relating the first property and the second property; and
means for associating the web page with a category based on a result of the evaluation,
wherein the means for evaluating a logical expression relating the first' property and the second property is configured to evaluate whether a constant value matches at least a portion of the content of the web page.
18. The apparatus of claim 17 , wherein the means for associating the web page with a category based on a result of the evaluation comprises one or more hardware processors.