Method and system for analyzing a user agent string
Method, system, and programs for analyzing user agent string are disclosed. In one example, a user agent string is received. A list of predefined keywords is obtained. Each of the predefined keywords is associated with a type of user agent information. One or more candidate keywords are extracted from the user agent string based on the list of predefined keywords. An extraction pattern is obtained for each of the one or more candidate keywords. A keyword is determined with a version from the one or more candidate keywords based, at least in part, on the extraction patterns for the one or more candidate keywords. The determined keyword with the version represents the type of user agent information in the received use agent string.
1. A method, implemented on at least one computing device each of which has at least one processor, storage, and a communication platform connected to a network for analyzing user agent string, the method comprising:
receiving a user agent string that identifies a user agent acting on behalf of a user device;
obtaining a list of keywords with metadata comprising user agent information associated with each keyword;
extracting one or more candidate keywords from the user agent string based on the list of keywords;
obtaining one or more extraction patterns for each of the one or more candidate keywords, the one or more extraction patterns including one or more predefined conditions for invoking version extraction patterns to extract a keyword version; and
determining a first keyword with a first keyword version from the one or more candidate keywords based, at least in part, on the one or more extraction patterns and the metadata, wherein determining the keyword and the first keyword version comprises:
extracting the first keyword version of the one or more candidate keywords from the user agent string based on one of the version extraction patterns being invoked in response to a corresponding one of the one or more predefined conditions being met, the first keyword version indicating a version of the first keyword included within the user agent string, and the first keyword is validated based on the metadata, wherein the invoked version extraction pattern indicates (i) a pattern of characters expected within the user agent string for the first keyword version and (ii) a position of where the first keyword version is to be located in the user agent string.
2. The method of claim 1 , wherein the user agent information includes at least one of operating system, browser, crawler, e-mail client, or game console.
3. The method of claim 1 , wherein:
each keyword of the list is associated with at least one condition for validating a corresponding candidate keyword, the at least one condition being indicated via the metadata associated with the keyword; and
the one or more candidate keywords are extracted based, at least in part, on the at least one condition.
4. The method of claim 3 , wherein the at least one condition includes utilizing one or more neighbor charsets of the corresponding candidate keyword in the user agent string.
5. The method of claim 1 , wherein:
each of the one or more candidate keywords is associated with a priority; and
the first keyword with the first keyword version is determined based, at least in part, on priorities of the one or more candidate keywords.
6. The method of claim 1 , wherein the list of keywords and the extraction patterns are updated with new keywords, new extraction patterns, or new keywords and new extraction patterns.
7. A system having at least one processor storage, and a communication platform for analyzing user agent string, the system comprising:
a pre-processing module configured for receiving a user agent string that identifies a user agent acting on behalf of a user;
a keyword fetching unit configured for obtaining a list of keywords with metadata comprising user agent information associated with each keyword;
a keyword extracting module configured for extracting one or more candidate keywords from the user agent string based on the list of keywords;
a pattern fetching unit configured for obtaining one or more extraction patterns for each of the one or more candidate keywords, the one or more extraction patterns including one or more predefined conditions for invoking version extraction patterns to extract a keyword version; and
an analyzing module configured for determining a first keyword with a first keyword version from the one or more candidate keywords based, at least in part, on the one or more extraction patterns and the metadata, wherein determining the first keyword and the first keyword version comprises:
extracting the first keyword version of the one or more candidate keywords from the user agent string based on one of the version extraction patterns being invoked in response to a corresponding one of the one or more predefined conditions being met, the first keyword version indicating a version of the first keyword included within the user agent string, and the first keyword is validated based on the metadata, wherein the invoked version extraction pattern indicates (i) a pattern of characters expected within the user agent string for the first keyword version and (ii) a position of where the first keyword version is to be located in the user agent string.
8. The system of claim 7 , wherein the user agent information includes at least one of operating system, browser, crawler, e-mail client, or game console.
9. The system of claim 7 , wherein:
each keyword of the list is associated with at least one condition for validating a corresponding candidate keyword, the at least one condition being indicated via the metadata associated with the keyword; and
the one or more candidate keywords are extracted based, at least in part, on the at least one condition.
10. The system of claim 9 , wherein the at least one condition includes utilizing one or more neighbor charsets of the corresponding candidate keyword in the user agent string.
11. The system of claim 7 , wherein:
each of the one or more candidate keywords is associated with a priority; and
the first keyword with the first keyword version is determined based, at least in part, on priorities of the one or more candidate keywords.
12. The system of claim 7 , wherein the list of keywords and the extraction patterns are updated with new keywords, new extraction patterns, or new keywords and new extraction patterns.
13. A non-transitory machine-readable medium having information recorded thereon for analyzing user agent string, wherein the information, when read by the machine, causes the machine to perform the following:
receiving a user agent string that identifies a user agent acting on behalf of a user;
obtaining a list of keywords with metadata comprising user agent information associated with each keyword;
extracting one or more candidate keywords from the user agent string based on the list of keywords;
obtaining one or more extraction patterns for each of the one or more candidate keywords, the one or more extraction patterns including one or more predefined conditions for invoking version extraction patterns to extract a keyword version; and
determining a first keyword with a first keyword version from the one or more candidate keywords based, at least in part, on the one or more extraction patterns and the metadata, wherein determining the first keyword and the first keyword version comprises:
extracting the first keyword version of the one or more candidate keywords from the user agent string based on one of the version extraction patterns being invoked in response to a corresponding one of the one or more predefined conditions being met, the first keyword version indicating a version of the first keyword included within the user agent string, and the first keyword is validated based on the metadata, wherein the invoked version extraction pattern indicates (i) a pattern of characters expected within the user agent string for the first keyword version and (ii) a position of where the first keyword version is to be located in the user agent string.
14. The medium of claim 13 , wherein:
each keyword of the list is associated with at least one condition for validating a corresponding candidate keyword, the at least one condition being indicated via the metadata associated with the keyword; and
the one or more candidate keywords are extracted based, at least in part, on the at least one condition.
15. The medium of claim 14 , wherein:
the at least one condition includes utilizing one or more neighbor charsets of the corresponding candidate keyword in the user agent string; and
the user agent information includes at least one of operating system, browser, crawler, e-mail client, or game console.
16. The medium of claim 13 , wherein:
each of the one or more candidate keywords is associated with a priority; and
the first keyword with the first keyword version is determined based, at least in part, on priorities of the one or more candidate keywords.
17. The medium of claim 13 , wherein the list of keywords and the extraction patterns are updated with new keywords, new extraction patterns, or new keywords and new extraction patterns.