Message phishing detection using machine learning characterization
An email phishing detection mechanism is provided that utilizes machine learning algorithms. The machine learning algorithms are trained on phishing and non-phishing features extracted from a variety of data sets. Embodiments extract embedded URL-based and email body text-based feature sets for training and testing the machine learning algorithms. Embodiments determine the presence of a phishing message through a combination of examining an embedded URL and the body text of the message for the learned feature sets.
1. An information handling system configured as an electronic mail server for an enterprise network, and comprising:
a processor;
a network interface, coupled to the processor, and communicatively coupled to the enterprise network;
a first memory, storing instructions executable by the processor and configured to
extract a uniform resource locator (URL) address embedded in an electronic mail (email) message received by the network interface,
determine whether the extracted URL comprises one or more features associated with a phishing URL, the one or more features associated with a phishing URL being pre-determined by training a machine-learning classifier,
extract body text from the email message,
determine whether the extracted body text comprises one or more features associated with a phishing email message, the one or more features associated with a phishing URL being pre-determined by training the machine-learning classifier, and
classify the email message as one of phishing or not phishing using the determinations associated with the extracted URL and the extracted body text.
2. The information handling system of claim 1 , wherein the machine-learning classifier comprises a machine-learning URL classifier; and,
the one or more features associated with a phishing URL are pre-determined by training the machine-learning URL classifier on one or more datasets comprising known phishing URLs and known non-phishing URLs.
3. The information handling system of claim 2 , wherein the one or more features associated with a phishing URL comprise one or more of:
use of shortening services on the URL;
an Internet Protocol (IP) address;
a URL of greater than 75 characters;
an “@” symbol within the URL;
multiple sets of double slashes within the URL;
a prefix or suffix separated by a hyphen to a domain of the URL;
one or more subdomains; and
an “https” within the domain of the URL.
4. The information handling system of claim 1 , wherein the machine-learning classifier comprises a machine-learning body text classifier, and,
the one or more features associated with a phishing email message are pre-determined by training the machine-learning body text classifier on one or more datasets comprising known phishing emails and known non-phishing emails.
5. The information handling system of claim 4 , wherein the one or more features associated with a phishing email message comprise one or more of:
a general greeting;
a lack of richness in vocabulary;
a lack of similarity between a subject of the email and the extracted body text;
excessive use of pronoun references; and
intent of the body text that is indicative of notification, urgency, action, consequence, and threat.
6. The information handling system of claim 1 wherein the processor is configured to determine whether the extracted body text comprises one or more features associated with a phishing email message by being further configured to:
perform natural language processing of the extracted body text to generate dependency and part-of-speech tags associated with sentences within the extracted body text; and
use the tags to determine whether the extracted body text comprises language constructs associated with an intent indicative of notification, urgency, action, consequence, and threat.
7. The information handling system of claim 6 wherein the processor is configured to determine whether the extracted body text comprises the language constructs by being further configured to:
match the tags with language construct information stored in a second memory coupled to the processor in one or more dictionaries, wherein the language construct information stored in the dictionaries is pre-determined during training of a machine-learning body text classifier on one or more datasets comprising known phishing emails and known non-phishing emails.
8. The information handling system of claim 1 wherein the processor is configured to classify the email message as one of phishing or not phishing by being further configured to:
classify the email message as a phishing message when both the extracted URL phishing determination and the extracted body text determination indicate the message is phishing;
classify the email message as a phishing message when one of the extracted URL phishing determination or the extracted body text determination has a probability of being phishing above a set threshold;
classify the email message as not a phishing message when neither of the extracted URL phishing determination or the extracted body text determination has a probability of being phishing above the set threshold.
9. The information handling system of claim 1 wherein the instructions executable by the processor are further configured to
store the email message in a third memory; and
transmit a quarantine message to a recipient of the email message, wherein the quarantine message comprises a notification that the email message has been quarantined, and information associated with the email message.
10. The information handling system of claim 1 , wherein the determining whether the extracted URL comprises one or more features associated
with a phishing URL and the determining whether the extracted body text comprises one or more features associated with a phishing email message are both performed by an associated machine learning classifier.
11. A method for identifying phishing email messages, the method comprising:
receiving, at a network interface coupled to an enterprise network, an electronic mail (email) message;
extracting a uniform resource locator (URL) address embedded in the email message;
determining whether the extracted URL comprises one or more features associated with a phishing URL, the one or more features associated with a phishing URL being pre-determined by training a machine-learning classifier;
extracting body text from the email message;
determining whether the extracted body text comprises one or more features associated with a phishing email message, the one or more features associated with a phishing URL being pre-determined by training the machine-learning classifier; and
classifying the email message as one of phishing or not phishing using the determinations associated with the extracted URL and the extracted body text.
12. The method of claim 11 wherein
the machine-learning classifier comprises a machine-learning URL classifier; and,
the one or more features associated with a phishing URL are pre-determined by training the machine-learning URL classifier on one or more datasets comprising known phishing URLs and known non-phishing URLs.
13. The method of claim 12 , wherein the one or more features associated with a phishing URL comprise one or more of:
use of shortening services on the URL;
an Internet Protocol (IP) address;
a URL of greater than 75 characters;
an “@” symbol within the URL;
multiple sets of double slashes within the URL;
a prefix or suffix separated by a hyphen to a domain of the URL;
one or more subdomains; and
an “https” within the domain of the URL.
14. The method of claim 11 wherein
the machine-learning classifier comprises a machine-learning body text classifier; and,
the one or more features associated with a phishing email message are pre-determined by training the machine-learning body text classifier on one or more datasets comprising known phishing emails and known non-phishing emails.
15. The method of claim 14 , wherein the one or more features associated with a phishing email message comprise one or more of:
a general greeting;
a lack of richness in vocabulary;
a lack of similarity between a subject of the email and the extracted body text;
excessive use of pronoun references; and
intent of the body text that is indicative of notification, urgency, action, consequence, and threat.
16. The method of claim 11 wherein said determining whether the extracted body text comprises one or more features associated with a phishing email message further comprises:
performing natural language processing of the extracted body text to generate dependency and part-of-speech tags associated with sentences within the extracted body text; and
determining whether the extracted body text comprises language constructs associated with an intent indicative of notification, urgency, action, consequence, and threat, using the tags associated with the sentences.
17. The method of claim 16 wherein said determining whether the extracted body text comprises the language constructs further comprises:
matching the tags with language construct information stored in a second memory coupled to the processor in one or more dictionaries, wherein the language construct information stored in the dictionaries is pre-determined during training of a machine-learning body text classifier on one or more datasets comprising known phishing emails and known non-phishing emails.
18. The method of claim 11 wherein said classifying the email message as one of phishing or not phishing further comprises:
classifying the email message as a phishing message when both the extracted URL phishing determination and the extracted body text determination indicate the message is phishing;
classifying the email message as a phishing message when one of the extracted URL phishing determination or the extracted body text determination has a probability of being phishing above a set threshold;
classifying the email message as not a phishing message when neither of the extracted URL phishing determination or the extracted body text determination has a probability of being phishing above the set threshold.
19. The method of claim 11 further comprising:
storing the email message in a third memory; and
transmitting a quarantine message to a recipient of the email message, using the network interface, wherein the quarantine message comprises a notification that the email message has been quarantined, and information associated with the email message.
20. An information handling system configured to examine communications incoming to an enterprise network for phishing communications, and comprising:
a processor;
a network interface, coupled to the processor, and communicatively coupled to the enterprise network;
a first memory storing instructions executable by the processor and configured to
extract a uniform resource locator (URL) address embedded in an incoming communication message received by the network interface,
determine whether the extracted URL comprises one or more features associated with a phishing URL, the one or more features associated with a phishing URL being pre-determined by training a machine-learning classifier,
extract body text from the incoming communication message,
determine whether the extracted body text comprises one or more features associated with a phishing communication, the one or more features associated with a phishing URL being pre-determined by training the machine-learning classifier, and
classify the incoming communication message as one of phishing or not phishing using the determinations associated with the extracted URL and the extracted body text.