Automated collection of branded training data for security awareness training
A method of collecting training data related to a branded phishing URL may comprise retrieving a phishing URL impersonating a brand; fetching a final webpage referenced thereby; determining the main language of the textual content thereof; rendering graphical representation(s) of the final webpage; extracting, from the source of URLs, information including the retrieved phishing URL, a brand, a type and a date associated therewith and storing the extracted information together with the final webpage and the rendered graphical representation(s). A message that contains a URL matching the phishing URL may then be retrieved. The main language of the textual content of the message may be determined and graphical representations thereof rendered. A record may be updated with the message, the main language and the rendered graphical representations, which may be made accessible as training data to train users to recognize phishing websites and messages.
1. A computer-implemented method of collecting training data related to a branded phishing Uniform Resource Locator (URL), comprising:
retrieving, over a computer network, a phishing URL impersonating a brand from a source of URLs;
fetching a final webpage referenced by the retrieved phishing URL;
determining a main language of a textual content of the fetched final webpage;
rendering at least one graphical representation of the fetched final webpage,
extracting, from the source of URLs, information including the retrieved phishing URL, a brand associated with the retrieved phishing URL, a type associated with the retrieved phishing URL, a date associated with the retrieved phishing URL and storing the extracted information in a database over the computer network, together with the fetched final webpage, a determined main language of the fetched final webpage, and the rendered at least one graphical representations of the fetched final webpage;
retrieving, from a source of messages, a message that contains a URL matching the retrieved phishing URL;
determining a main language of a textual content of the retrieved message;
rendering at least one graphical representation of the retrieved message;
updating a record in a database with the message, the determined main language of the message, and the at least one rendered graphical representation of the message; and
making the stored extracted information and the updated record accessible over the computer network at least as training data to train end users to recognize phishing websites and phishing messages.
2. The computer-implemented method of claim 1 , wherein fetching the final webpage comprises following at least one redirection leading to the final webpage.
3. The computer-implemented method of claim 2 , wherein the at least one redirection comprises at least one of an HTTP status code 3xx redirection, an HTML meta refresh redirection and a programming language redirection.
4. The computer-implemented method of claim 1 , wherein the graphical representations of the fetched final webpage comprise metadata associated with the fetched final webpage.
5. The computer-implemented method of claim 4 , wherein the metadata comprises a phishing URL.
6. The computer-implemented method of claim 1 , wherein the graphical representations of the fetched final webpage comprise at least some of a graphical representation of the fetched final webpage on a mobile device, a graphical representation of the fetched final webpage on a tablet computer, a graphical representation of the fetched final webpage on a laptop computer and a graphical representation of the fetched final webpage on a desktop computer.
7. The computer-implemented method of claim 1 , wherein each of the graphical representations of the fetched final webpage comprise a predetermined width and height.
8. The computer-implemented method of claim 1 , wherein the retrieved message comprises one of an email, a text message and an instant message.
9. The computer-implemented method of claim 1 , further comprising at least one of prior parsing, decoding, decompressing and decrypting a content of the retrieved message.
10. The computer-implemented method of claim 1 , wherein the matching of the URL retrieved from the source of messages with the retrieved phishing URL requires an exact match.
11. The computer-implemented method of claim 1 , wherein the matching of the URL retrieved from the source of messages with the retrieved phishing URL requires fewer than all URL elements to match.
12. The computer-implemented method of claim 11 , wherein the URL elements comprise at least one of a label of the domain name, an element of the path, an element of the query string and the fragment.
13. The computer-implemented method of claim 11 , wherein the matching of URL elements of the URL retrieved from the source of messages with the retrieved phishing URL is one of case sensitive and case insensitive.
14. The computer-implemented method of claim 1 , wherein the graphical representations of the retrieved message comprise metadata associated with the retrieved message.
15. The computer-implemented method of claim 14 , wherein the metadata of the retrieved message comprises at least one of a sender of the message, a date when the message was sent and a subject of the retrieved message.
16. The computer-implemented method of claim 1 , wherein the graphical representations of the retrieved message comprise at least some of: a graphical representation of the retrieved message on a mobile device, a graphical representation of the retrieved message on a tablet computer, a graphical representation of the retrieved message on a laptop computer and a graphical representation of the retrieved message on a desktop computer.
17. The computer-implemented method of claim 1 , wherein each of the graphical representations of the retrieved message comprises a predetermined width and height.
18. A computing device configured to collect training data related to a branded phishing Uniform Resource Locator (URL), comprising:
at least one processor;
at least one data storage device coupled to the at least one processor;
a network interface coupled to the at least one processor and to a computer network;
a plurality of processes spawned by the at least one processor, the processes including processing logic for:
retrieving, over a computer network, a phishing URL impersonating a brand from a source of URLs;
fetching a final webpage referenced by the retrieved phishing URL;
determining a main language of a textual content of the fetched final webpage;
rendering at least one graphical representation of the fetched final webpage,
extracting, from the source of URLs, information including the retrieved phishing URL, a brand associated with the retrieved phishing URL, a type associated with the retrieved phishing URL, a date associated with the retrieved phishing URL and storing the extracted information in a database over the computer network, together with the fetched final webpage, a determined main language of the fetched final webpage, and the rendered at least one graphical representations of the fetched final webpage;
retrieving, from a source of messages, a message that contains a URL matching the retrieved phishing URL;
determining a main language of a textual content of the retrieved message;
rendering at least one graphical representation of the retrieved message;
updating a record in a database with the message, the main language of the message, and the at least one rendered graphical representation of the retrieved message; and
making the stored extracted information and the updated record accessible over the computer network at least as training data to train end users to recognize phishing websites and phishing messages.
19. The computing device of claim 18 , wherein the processing logic for fetching the final webpage comprises processing logic to follow at least one redirection leading to the final webpage.
20. The computing device of claim 19 , wherein the at least one redirection comprises at least one of an HTTP status code 3xx redirection, an HTML meta refresh redirection and a programming language redirection.
21. The computing device of claim 18 , wherein at least one of the graphical representations of the fetched final webpage comprises metadata associated with the fetched final webpage.
22. The computing device of claim 21 , wherein the metadata comprises a phishing URL.
23. The computing device of claim 18 , wherein the graphical representations of the fetched final webpage comprise at least some of a graphical representation of the fetched final webpage on a mobile device, a graphical representation of the fetched final webpage on a tablet computer, a graphical representation of the fetched final webpage on a laptop computer and a graphical representation of the fetched final webpage on a desktop computer.
24. The computing device of claim 18 , wherein each of the graphical representations of the fetched final webpage comprise a predetermined width and height.
25. The computing device of claim 18 , wherein the retrieved message comprises one of an email, a text message and an instant message.
26. The computing device of claim 18 , further comprising processing logic to at least one of parse, decode, decompress and decrypt a content of the retrieved message.
27. The computing device of claim 18 , wherein the processing logic for matching the URL retrieved from the source of messages with the retrieved phishing URL requires an exact match.
28. The computing device of claim 18 , wherein the processing logic for matching the URL retrieved from the source of messages with the retrieved phishing URL allows for fewer than all URL elements to match.
29. The computing device of claim 28 , wherein the URL elements comprise at least one of a label of the domain name, an element of the path, an element of the query string and the fragment.
30. The computing device of claim 28 , wherein processing logic for matching of the URL retrieved from the source of messages and the retrieved phishing URL is configured to be one of case sensitive and case insensitive.
31. The computing device of claim 18 , wherein the graphical representations of the retrieved message comprise metadata associated with the retrieved message.
32. The computing device of claim 31 , wherein the metadata of the retrieved message comprises at least one of a sender of the message, a date when the message was sent and a subject of the retrieved message.
33. The computing device of claim 18 , wherein the graphical representations of the retrieved message comprise at least some of a graphical representation of the retrieved message on a mobile device, a graphical representation of the retrieved message on a tablet computer, a graphical representation of the retrieved message on a laptop computer and a graphical representation of the retrieved message on a desktop computer.
34. The computing device of claim 18 , wherein each of the graphical representations of the retrieved message comprises a predetermined width and height.