System and method for analyzing web content
View Patent ↗A system and method are provided for identifying active content in websites on a network. One embodiment includes a method of classifying web content. In one embodiment, the classifications are indicative of active and/or malicious content. The method includes identifying properties associated with the web page based at least partly on the content of the web page and storing said properties in a database of web page properties. The method further includes comparing at least one definition to properties stored in the database of web page properties and identifying the web page with at least one definition based on comparing said definition with said stored properties. The method further includes identifying the web page with at least one category associated with the at least one definition, wherein said category is indicative of active content associated with the web page. Other embodiments include systems configured to perform such methods.
1. A method of classifying web content, implemented on one or more computer processors, the method comprising:
using at least one of the processors, receiving content of at least one web page;
using at least one of the processors, determining a first property of a plurality of properties associated with the web page at least in part by identifying a keyword in the content of the web page, the first property comprising static content associated with the web page,
using at least one of the processors, determining a second property of the plurality of properties at least in part by executing active content associated with the webpage in a sandbox environment, the second property comprising the active content;
using at least one of the processors, storing the first property and the second property in a database of web page properties;
using at least one of the processors, comparing at least one definition from a definitions database to the first and second properties, wherein a particular definition contains a logical expression relating the first property and the second property;
using at least one of the processors, associating the web page with at least one category, the category associated with the particular definition, wherein said category is indicative of the active content associated with the web page;
wherein the logical expression comprises at least one term comprising a relationship of at least one web page property to at least one other value; and
wherein the at least one other value comprises a constant value matching at least a portion of the content of the web page.
2. The method of claim 1 , wherein comparing at least one definition from a definitions database comprises performing at least one database query associated with the at least one definition, wherein the query selects the web page from the database of web page properties based at least partly on the properties of the selected at least one web page.
3. The method of claim 1 , wherein the determining a first property of a plurality of properties is performed using a first processor and the executing active content associated with the webpage in a sandbox environment is performed using a second processor.
4. The method of claim 1 , wherein associating the web page with a category, the category associated with the at least one definition comprises storing data associating a uniform resource locator of the at least one of the web pages with the category.
5. The method of claim 1 , wherein the category identifies the at least one web page as having malicious content.
6. The method of claim 1 , further comprising receiving the at least one definition from a database of definitions.
7. The method of claim 1 , wherein the at least one other value comprises at least one other web page property matching at least a portion of the content of the web page.
8. The method of claim 1 , wherein the first property is associated with a string matching at least a portion of the content of the web page.
9. The method of claim 1 , wherein the first property is associated with a regular expression matching at least a portion of the content of the web page.
10. The method of claim 8 , wherein the first property comprises a number indicative of occurrences within the content of the web page.
11. The method of claim 8 , further comprising determining a score associated with the URL of the web page, wherein identifying the web page with at least one category is based at least partly on the score.
12. The method of claim 1 , further comprising storing data indicative of a URL of the web page to said database, wherein said database associates the properties identified based at least partly on the content of the web page to said URL of the web page.
13. A system for classifying web content, the system comprising:
a database configured to store properties associated with web pages;
one or more processing devices configured, individually or in combination, to:
determine a first property of a plurality of properties of a web page, at least in part by identifying a keyword in the content of the web page, the first property comprising static content associated with the web page;
determine a second property of the plurality of properties, at least in part by executing active content associated with the webpage in a sandbox environment, the second property comprising the active content;
store said plurality of properties in said database of web page properties;
compare at least one definition from a definitions database to the first and second properties, wherein a particular definition contains a logical expression relating the first property and the second property;
associate the web page with at least one category, the category associated with the particular definition, wherein said category is indicative of active content associated with the web page;
wherein the logical expression comprises at least one term comprising a relationship of at least one web page property to at least one other value; and
wherein the at least one other value comprises a constant value.
14. The system of claim 13 , wherein the one or more processing devices are configured to compare the first property and the second property to the definition at least in part by performing at least one database query associated with at least one definition, wherein the query selects the web page from the database of web page properties based at least partly on the first property and the second property.
15. The system of claim 13 , wherein the determining a first property is performed using a first processor and the execution of active content associated with the webpage in a sandbox environment is performed using a second processor.
16. The system of claim 13 , wherein associating the web page with at least one category associated with the definition comprises storing data associating a uniform resource locator of the at least one of the web pages with the category.
17. The system of claim 13 , wherein the category identifies the at least one of the web pages as having malicious content.
18. The system of claim 13 , wherein the at least one other value comprises at least one other web page property.
19. The system of claim 13 , wherein the first property is associated with a string matching at least a portion of the content of the web page.
20. The system of claim 13 , wherein the first property is associated with a regular expression matching at least a portion of the content of the web page.
21. The system of claim 13 , wherein the first property comprises a number indicative of occurrences within the content of the web page.
22. The system of claim 13 , wherein said database of web page properties is configured to store data indicative of a URL of the web page and wherein said database associates the properties identified based at least partly on the content of the web page to said URL of the web page.