System and method for internet endpoint profiling

1. A method of profiling an Internet endpoint associated with an Internet Protocol (IP) address, comprising:

generating a profiling rule using an Internet search engine;

obtaining a search result by inputting the IP address as a search phrase to the Internet search engine; and

classifying the Internet endpoint based on the search result using the profiling rule,

wherein the profiling rule comprises a IP tag and a key phrase list having a phrase associated with a URL class, the URL class being associated with the IP tag,

wherein the profiling rule further comprises a data structure having an entry, the entry having a first domain name associated with the phrase,

wherein the data structure is a hash table having a plurality of indexes corresponding to a plurality of values, the plurality of indexes comprising the first domain name, the plurality of values comprising the phrase,

wherein the search result comprises:

a Uniform Resource Locator (URL) having a second domain name; and

a hit text associated with the URL, the hit text comprising a phrase in the key phrase list,

wherein classifying the Internet endpoint comprises assigning the URL class to the URL and assigning the IP tag to the Internet end point if the first domain name is the same as the second domain name,

wherein generating the profiling rule comprises:

adding an index comprising the second domain name to the hash table if the plurality of indexes do not comprise the second domain name;

setting a counter to an initial count;

obtaining another search result by inputting another IP address to the Internet search engine, the another search result comprising another hit text associated with another URL; the another hit text comprising another phrase in the key phrase list;

incrementing the counter if the another URL comprises the second domain name; and

setting a value in the hash table corresponding to the index based on at least one selected from the group consisting of the phrase and the another phrase if the counter exceeds a pre-determined threshold.

2. A computer readable medium, embodying instructions executable by the computer to perform method steps for profiling an Internet endpoint associated with an Internet Protocol (IP) address, the instructions comprising functionality for:

generating a profiling rule using an Internet search engine;

obtaining a search result by inputting the IP address as a search phrase to the Internet search engine; and

classifying the Internet endpoint based on the search result using the profiling rule,

wherein the profiling rule comprises a IP tag and a key phrase list having a phrase associated with a URL class, the URL class being associated with the IP tag,

wherein the profiling rule further comprises a data structure having an entry, the entry having a first domain name associated with the phrase,

wherein the data structure is a hash table having a plurality of indexes corresponding to a plurality of values, the plurality of indexes comprising the first domain name, the plurality of values comprising the phrase,

wherein the search result comprises:

a Uniform Resource Locator (URL) having a second domain name; and

a hit text associated with the URL, the hit text comprising a phrase in the key phrase list,

wherein classifying the Internet endpoint comprises assign the URL class to the URL and assigning the IP tag to the Internet end point if the first domain name is the same as the second domain name,

wherein generating the profiling rule comprises:

adding an index comprising the second domain name to the hash table if the plurality of indexes do not comprise the second domain name;

setting a counter to an initial count;

obtaining another search result by inputting another IP address to the Internet search engine, the another search result comprising another hit text associated with another URL; the another hit text comprising another phrase in the key phrase list;

incrementing the counter if the another URL comprises the second domain name; and

setting a value in the hash table corresponding to the index based on at least one selected from the group consisting of the phrase and the another phrase if the counter exceeds a pre-determined threshold.

3. The method of claim 1 , wherein generating the profiling rule further comprises:

obtaining a seed search result by inputting a seed set to the Internet search engine, the seed set comprising a plurality of randomly chosen IP addresses, the seed search result comprising a plurality of hit texts and another Uniform Resource Locator (URL) associated with another hit text of the plurality of hit texts, the plurality of hit texts comprising a plurality of phrases, the another hit text comprising another phrase of the plurality of phrases;

ranking the plurality of phrases to generate a rank of the another phrase based a count of the another phrase in the plurality of hit texts;

adding the another phrase to the key phrase list if the count exceeds another pre-determined threshold;

assigning another URL class to the another URL if the another phrase is added to the key phrase list, the another URL class being determined from the another phrase based on semantics;

determining another IP tag associated with the another URL class, the another IP tag being determined from the another URL class and the another phrase based on semantics; and

associating the another phrase with the another URL class in the profiling rule.

4. The method of claim 1 , wherein the Internet search engine comprises functionality for:

collecting a snapshot of a plurality of web-pages in at least a portion of Internet;

generating a reverse-index based on the snapshot, the reverse-index being a mapping of phrases to web-pages having the phrases therein; and

resolving a search query using the reverse-index to generate the search result, the search query comprising at least one selected from a group consisting of the search phrase and two or more search phrases combined via one or more logical operators.

5. The method of claim 1 ,

wherein classifying the Internet endpoint further comprises:

assigning the URL class to the URL and assigning the IP tag to the Internet end point if the second domain name comprises the phrase.

6. The method of claim 1 , further comprising:

identifying a plurality of top ranked endpoints from at least a portion of the Internet based on traffic flow distribution associated with at least the portion of the Internet;

classifying the plurality of top ranked endpoints using the profiling rule to generate a plurality of IP tags;

identifying a plurality of server tags from the plurality of IP tags based on semantics;

classifying a traffic flow having a source IP address and a destination IP address according to the plurality of server tags based on semantics, wherein an IP tag of at least one selected from a group consisting of the source IP address and the destination IP address matches at least one of the plurality of IP tags.

7. The computer readable medium of claim 2 , the instructions further comprising functionality for:

obtaining a seed search result by inputting a seed set to the Internet search engine, the seed set comprising a plurality of randomly chosen IP addresses, the seed search result comprising a plurality of hit texts and another Uniform Resource Locator (URL) associated with another hit text of the plurality of hit texts, the plurality of hit texts comprising a plurality of phrases, the another hit text comprising another phrase of the plurality of phrases;

ranking the plurality of phrases to generate a rank of the another phrase based a count of the another phrase in the plurality of hit texts;

adding the another phrase to the key phrase list if the count exceeds another pre-determined threshold;

assigning another URL class to the another URL if the another phrase is added to the key phrase list, the another URL class being determined from the another phrase based on semantics;

determining another IP tag associated with the another URL class, the another IP tag being determined from the another URL class and the another phrase based on semantics; and

associating the another phrase with the another URL class in the profiling rule.

8. The computer readable medium of claim 2 , wherein the Internet search engine comprises functionality for:

collecting a snapshot of a plurality of web-pages in at least a portion of Internet;

generating a reverse-index based on the snapshot, the reverse-index being a mapping of phrases to web-pages having the phrases therein; and

resolving a search query using the reverse-index to generate the search result, the search query comprising at least one selected from a group consisting of the search phrase and two or more search phrases combined via one or more logical operators.

9. The computer readable medium of claim 2 ,

the instructions further comprising functionality for:

assigning the URL class to the URL and assigning the IP tag to the Internet end point if the second domain name comprises the phrase.

10. The method of claim 3 , wherein the hit text comprises content of a webpage referenced by the URL.

11. The computer readable medium of claim 7 , wherein the hit text comprises content of a webpage referenced by the URL.