Granted Patent B2

US 10,025,847 · App. 14/410,702 · Granted Jul 17, 2018

Abstract

Claims (62)

1. A method, implemented on at least one computing device each of which has at least one processor, storage, and a communication platform connected to a network for determining a keyword from user agent strings, the method comprising:

receiving a plurality of user agent strings;

grouping the plurality of user agent strings into one or more clusters, wherein the one or more clusters comprise a first cluster that includes two or more user agent strings;

comparing the two or more user agent strings in the first cluster;

extracting a longest common subsequence among the two or more user agent strings;

removing the longest common subsequence from each user agent string to obtain a remaining subsequence; and

determining a keyword from the first cluster based on at least one of the longest common subsequence and the remaining subsequence, wherein the keyword represents a type of user agent information.

2. The method of claim 1 , wherein the grouping comprises:

assigning each user agent string into a cluster;

calculating a distance between each pair of clusters;

identifying a pair of clusters having a minimum distance;

merging the pair of clusters into one cluster if the minimum distance is less than a threshold; and

repeating the calculating, the identifying, and the merging, until a minimum distance exceeds the threshold to generate the one or more clusters.

3. The method of claim 2 , wherein the calculating comprises:

calculating a distance between each pair of user agent strings, each of which is from one of the pair of clusters to obtain calculated distances;

determining the distance between the pair of clusters based on a minimum distance among the calculated distances.

4. The method of claim 2 , wherein the threshold is predetermined or dynamically modified based on a machine learning model.

5. The method of claim 1 , further comprising:

ranking the merged clusters based on number of user agent strings in each cluster; and

selecting the one or more clusters from the merged clusters based on the ranking.

6. The method of claim 1 , wherein the type of user agent information include at least one of operating system, browser, crawler, e-mail client, and game console.

7. The method of claim 1 , furthering comprising:

providing the keyword to an administrator for confirmation; and

storing the keyword into a database along with the type of user agent information upon confirmation from the administrator.

8. A system having at least one processor storage, and a communication platform for determining a keyword from user agent strings, the system comprising:

a user agent receiver configured for receiving a plurality of user agent strings;

a user agent clustering unit configured for grouping the plurality of user agent strings into one or more clusters, wherein the one or more clusters comprise a first cluster that includes two or more user agent strings;

a user agent comparing unit configured for comparing the two or more user agent strings in the first cluster;

a subsequence extractor configured for extracting a longest common subsequence among the two or more user agent strings;

a subsequence removing unit configured for removing the longest common subsequence from each user agent string to obtain a remaining subsequence; and

a keyword determiner configured for determining a keyword from the first cluster based on at least one of the longest common subsequence and the remaining subsequence, wherein the keyword represents a type of user agent information.

9. The system of claim 8 , wherein the user agent clustering unit comprises:

a distance calculation unit configured for assigning each user agent string into a cluster and calculating a distance between each pair of clusters;

a cluster merging determiner configured for identifying a pair of clusters having a minimum distance; and

a cluster merging unit configured for merging the pair of clusters into one cluster if the minimum distance is less than a threshold, wherein the calculating, the identifying, and the merging are repeated until a minimum distance exceeds the threshold to generate the one or more clusters.

10. The system of claim 9 , wherein calculating a distance between each pair of clusters comprises:

calculating a distance between each pair of user agent strings, each of which is from one of the pair of clusters to obtain calculated distances; and

determining the distance between the pair of clusters based on a minimum distance among the calculated distances.

11. The system of claim 9 , wherein the threshold is predetermined or dynamically modified based on a machine learning model.

12. The system of claim 8 , further comprising:

a cluster ranking unit configured for ranking the merged clusters based on number of user agent strings in each cluster; and

a cluster filter configured for selecting the one or more clusters from the merged clusters based on the ranking.

13. The system of claim 8 , wherein the type of user agent information include at least one of operating system, browser, crawler, e-mail client, and game console.

14. A non-transitory machine-readable medium having information recorded thereon for determining a keyword from user agent strings, wherein the information, when read by the machine, causes the machine to perform the following:

receiving a plurality of user agent strings;

grouping the plurality of user agent strings into one or more clusters, wherein the one or more clusters comprise a first cluster that includes two or more user agent strings;

comparing the two or more user agent strings in the first cluster;

extracting a longest common subsequence among the two or more user agent strings;

removing the longest common subsequence from each user agent string to obtain a remaining subsequence; and

determining a keyword from the first cluster based on at least one of the longest common subsequence and the remaining subsequence, wherein the keyword represents a type of user agent information.

15. The medium of claim 14 , wherein the grouping comprises:

assigning each user agent string into a cluster;

calculating a distance between each pair of clusters;

identifying a pair of clusters having a minimum distance;

merging the pair of clusters into one cluster if the minimum distance is less than a threshold; and

repeating the calculating, the identifying, and the merging, until a minimum distance exceeds the threshold to generate the one or more clusters.

16. The medium of claim 15 , wherein the calculating comprises:

calculating a distance between each pair of user agent strings, each of which is from one of the pair of clusters to obtain calculated distances;

determining the distance between the pair of clusters based on a minimum distance among the calculated distances.

17. The medium of claim 14 , wherein the information, when read by the machine, further causes the machine to perform the following:

ranking the merged clusters based on number of user agent strings in each cluster; and

selecting the one or more clusters from the merged clusters based on the ranking.

Assignments (6)