Methods, systems and devices to mitigate the effects of side effect URLs in legitimate and phishing electronic messages
A computer-implemented method may comprise receiving an electronic message (such as a legitimate email or malicious phishing email, for example) over a computer network, the electronic message containing a uniform resource locator (URL). The URL in the received electronic message may be parsed and one or more original parameter may be identified therein. The type of the identified original parameter(s) may be determined. The identified original parameter(s) may then be transformed according to a parameter transformation rules, selected according to the determined type. to generate transformed parameter(s). The URL may then be reassembled by substituting the transformed parameter(s) for the original parameter(s). The website pointed to by the reassembled URL may then be accessed, over the computer network, using the transformed parameter(s) only if the reassembled URL meets a predetermined minimum criterion.
1. A computer-implemented method, comprising:
receiving and storing an electronic message, in a memory of a computing device coupled to a computer network, the electronic message containing a uniform resource locator (URL);
parsing the URL in the electronic message stored in the memory of the computing device and identifying at least one original parameter in the URL, the at least one original parameter comprising a sequence of characters;
determining a length of the at least one original parameter;
determining a statistical distribution of lowercase letters, uppercase letters and/or numbers of the at least one original parameter;
determining a type of the identified at least one original parameter, the determined type being one of a plurality of predetermined types of parameters only when the length of the at least one original parameter is determined to be at least a predetermined minimum length and when the statistical distribution is determined to be consistent with normal distributions of such lowercase letters, uppercase letters and/or numbers;
transforming the identified at least one original parameter according to one of a plurality of parameter transformation rules selected according to the determined type to generate at least one transformed parameter;
reassembling the URL by substituting the at least one transformed parameter for the at least one original parameter;
accessing, over the computer network, the website pointed to by the reassembled URL using the at least one transformed parameter if the reassembled URL meets a predetermined minimum criterion,
foregoing accessing the reassembled URL if the reassembled URL does not meet the predetermined minimum criterion; and
analyzing a response of the accessed website to the at least one transformed parameters to determine whether the URL is a side effect URL.
2. The computer-implemented method of claim 1 , wherein the minimum criterion includes a predetermined minimum of transformations carried out on the at least one original parameter to generate the at least one transformed parameter.
3. The computer-implemented method of claim 1 , wherein the at least one original parameter is located in at least one of a query string and a path of the URL.
4. The computer-implemented method of claim 1 , wherein the plurality of predetermined types of parameters include at least some of decimal numbers, lowercase hexadecimal characters, uppercase hexadecimal characters, base 64 characters, lowercase letters, uppercase letters, lowercase email addresses and uppercase email addresses.
5. The computer-implemented method of claim 4 , wherein at least one of the parameter transformation rules specifies to replace each character of an original parameter to which the parameter transformation rule is applied with a random character of a same type.
6. The computer-implemented method of claim 4 , wherein at least one of the parameter transformation rules specifies to replace each character of an original parameter to which the parameter transformation rule is applied with a replacement character determined according to a predetermined operation.
7. The computer-implemented method of claim 1 , further including applying a selectable margin of error to the statistical distribution for each of the plurality of types of parameters.
8. A computing device configured to determine whether a received electronic message is suspect, comprising:
at least one processor;
at least one data storage device coupled to the at least one processor;
a network interface coupled to the at least one processor and to a computer network;
a plurality of processes spawned by said at least one processor, the processes including processing logic for:
using the network interface, receiving an electronic message;
storing the received electronic message in the data storage device, the stored electronic message containing a uniform resource locator (URL);
parsing the URL in the electronic message stored in the data storage device and identifying at least one original parameter in the URL, the at least one original parameter comprising a sequence of characters;
determining a length of the at least one original parameter;
determining a statistical distribution of lowercase letters, uppercase letters and/or numbers of the at least one original parameter;
determining a type of the identified at least one original parameter, the determined type being one of a plurality of predetermined types of parameters only when the length of the at least one original parameter is determined to be at least a predetermined minimum length and when the statistical distribution is determined to be consistent with normal distributions of such lowercase letters, uppercase letters and/or numbers;
transforming the identified at least one original parameter according to one of a plurality of parameter transformation rules selected according to the determined type to generate at least one transformed parameter;
reassembling the URL by substituting the at least one transformed parameter for the at least one original parameter;
accessing, over the computer network, the website pointed to by the reassembled URL using the at least one transformed parameter if the reassembled URL meets a predetermined minimum criterion, and
foregoing accessing the reassembled URL if the reassembled URL does not meet the predetermined minimum criterion; and
analyzing a response of the accessed website to the at least one transformed parameters to determine whether the URL is a side effect URL.
9. The computing device of claim 8 , wherein the minimum criterion includes a predetermined minimum of transformations carried out on the at least one original parameter to generate the at least one transformed parameter.
10. The computing device of claim 8 , wherein the at least one original parameter is located in at least one of a query string and a path of the URL.
11. The computing device of claim 8 , wherein the plurality of predetermined types of parameters include at least some of decimal numbers, lowercase hexadecimal characters, uppercase hexadecimal characters, base 64 characters, lowercase letters, uppercase letters, lowercase email addresses and uppercase email addresses.
12. The computing device of claim 11 , wherein at least one of the parameter transformation rules specifies to replace each character of an original parameter to which the parameter transformation rule is applied with a random character of a same type.
13. The computing device of claim 11 , wherein at least one of the parameter transformation rules specifies to replace each character of an original parameter to which the parameter transformation rule is applied with a replacement character determined according to a predetermined operation.
14. The computing device of claim 8 , further including applying a selectable margin of error to the statistical distribution for each of the plurality of types of parameters.
15. A tangible, non-transitory machine-readable data storage device having data stored thereon representing sequences of instructions which, when executed by a computing device, cause the computing device to:
receive and storing an electronic message, in a memory of a computing device coupled to a computer network, the electronic message containing a uniform resource locator (URL);
parse the URL in the electronic message stored in the memory of the computing device and identify at least one original parameter in the URL, the at least one original parameter comprising a sequence of characters;
determine a length of the at least one original parameter;
determine a statistical distribution of lowercase letters, uppercase letters and/or numbers of the at least one original parameter;
determine a type of the identified at least one original parameter, the determined type being one of a plurality of predetermined types of parameters only when the length of the at least one original parameter is determined to be at least a predetermined minimum length and when the statistical distribution is determined to be consistent with normal distributions of such lowercase letters, uppercase letters and/or numbers;
transform the identified at least one original parameter according to one of a plurality of parameter transformation rules selected according to the determined type to generate at least one transformed parameter;
reassemble the URL by substituting the at least one transformed parameter for the at least one original parameter;
access, over the computer network, the website pointed to by the reassembled URL using the at least one transformed parameter if the reassembled URL meets a predetermined minimum criterion, and
forego accessing the reassembled URL if the reassembled URL does not meet the predetermined minimum criterion; and
analyzing a response of the accessed website to the at least one transformed parameters to determine whether the URL is a side effect URL.
16. The tangible, non-transitory machine-readable data storage device of claim 15 , wherein the minimum criterion includes a predetermined minimum of transformations carried out on the at least one original parameter to generate the at least one transformed parameter.
17. The tangible, non-transitory machine-readable data storage device of claim 15 , wherein the at least one original parameter is located in at least one of a query string and a path of the URL.
18. The tangible, non-transitory machine-readable data storage device of claim 15 , wherein the plurality of predetermined types of parameters include at least some of decimal numbers, lowercase hexadecimal characters, uppercase hexadecimal characters, base 64 characters, lowercase letters, uppercase letters, lowercase email addresses and uppercase email addresses.
19. The tangible, non-transitory machine-readable data storage device of claim 18 , wherein at least one of the parameter transformation rules specifies to replace each character of an original parameter to which the parameter transformation rule is applied with a random character of a same type.
20. The tangible, non-transitory machine-readable data storage device of claim 18 , wherein at least one of the parameter transformation rules specifies to replace each character of an original parameter to which the parameter transformation rule is applied with a replacement character determined according to a predetermined operation.
21. The tangible, non-transitory machine-readable data storage device of claim 18 , further including sequences of instructions which, when executed by the computing device, cause the computing device to apply a selectable margin of error to the statistical distribution for each of the plurality of types of parameters.