Phishing detection enhanced through machine learning techniques
Phishing enhancement and phishing detection enhancement technologies. The technologies can include determinations of an effectiveness rate of one or more phishing threat actors. The technologies can also include selection of effective URLs from at least one effective phishing threat actor. The technologies can also include generation or adjustment of a phishing system using a machine learning process to identify patterns in the selected effective URLs that enable the selected effective URLs to avoid detection by the phishing detection system. The technologies can also include generation of synthetic phishing URLs using the phishing system and the identified patterns. The technologies can also include adjustments or training of the phishing system or the phishing detection system according to the synthetic phishing URLs to enhance the systems.
1. A method, comprising:
generating a phishing system using a machine learning process using a long short-term memory (LSTM) network to identify patterns in effective URLs that enable the effective URLs to avoid detection as phishing threats by a phishing detection system, wherein the effective URLs are from at least one effective phishing threat actor of a set of phishing threat actors, and wherein the effective URLs are URLs that avoid detection as phishing threats by the phishing detection system;
generating, by the machine learning process, a set of first artificial intelligence (AI) functions comprising a set of neurons that connect the machine learning process to a second machine learning process; and
generating, by the second machine learning process, a second AI function that classifies the effective URLs, wherein the second AI function is a sigmoid neuron;
generating synthetic phishing URLs using the generated phishing system and the identified patterns; and
adjusting the phishing detection system to recognize the synthetic phishing URLs such that the phishing detection system is enhanced by the generated phishing system.
2. The method of claim 1 , further comprising:
selecting new effective URLs from at least one effective phishing threat actor of the set of actors after adjusting the phishing detection system to recognize the synthetic phishing URLs;
adjusting the phishing system using the machine learning process to identify second patterns in the new effective URLs that enable the new effective URLs to avoid detection as phishing threats by the adjusted phishing detection system;
generating second synthetic phishing URLs using the adjusted phishing system and the identified second patterns; and
re-adjusting the adjusted phishing detection system to recognize the second synthetic phishing URLs such that the adjusted phishing detection system is further enhanced by the adjusted phishing system.
3. The method of claim 1 , wherein the machine learning process to identify patterns in the effective URLs comprises using a long short-term memory (LSTM) network.
4. The method of claim 1 , wherein the phishing detection system comprises a proactive detection system that uses a classification model that classifies URLs, and wherein the proactive detection system comprises using a long short-term memory (LSTM) network.
5. The method of claim 1 , wherein generating the phishing system comprises enhancing an effectiveness rate of a phishing attack by the at least one effective phishing threat actor, wherein the effectiveness rate of a phishing attack is defined as a ratio between the number of URLs that avoid detection as a phishing threat by the detection system and the number of total known URLs generated by the at least one effective phishing threat actor.
6. The method of claim 1 , wherein generating the phishing system comprises enhancing a success rate of a phishing attack by the at least one effective phishing threat actor, wherein the success rate of a phishing attack is defined by a ratio between the number of URLs that lead to retrieval of user credentials and the number of total known URLs generated by the at least one effective phishing threat actor.
7. The method of claim 1 , wherein generating the phishing system comprises enhancing an operational efficiency rate of a phishing attack by the at least one effective phishing threat actor, wherein the operational efficiency rate is defined by a ratio between the number of total known URLs generated by the at least one effective phishing threat actor and the amount of time spent to generate the total known URLs generated by the at least one effective phishing threat actor.
8. The method of claim 1 , wherein the generating the phishing system comprises an initialization phase, comprising:
generating a vocabulary of the at least one effective phishing threat actor based on the effective URLs; and
generating sentences based on the effective URLs.
9. The method of claim 8 , wherein the generating the phishing system comprises an encoding phase, comprising generating a one-hot encoding representation of the effective URLs based on the generated vocabulary and the generated sentences.
10. The method of claim 9 , wherein the generating the phishing system comprises a training phase, comprising:
receiving, by the machine learning process, the one-hot encoding representation.
11. The method of claim 1 , wherein the LSTM network is trained by back-propagation using a categorical cross-entropy loss function and a root mean square propagation optimizer to prevent oscillations during the training phase.
12. The method of claim 8 , wherein generating the sentences comprises:
collecting and concatenating the effective URLs into a full prose text; and
generating the sentences from the full prose text, wherein the sentences have lengths L by taking steps of size S.
13. The method of claim 9 , wherein generating the sentences comprises:
collecting and concatenating the effective URLs into a full prose text; and
generating the sentences from the full prose text, wherein the sentences have lengths L by taking steps of size S, and
wherein generating the one-hot encoding representation comprises generating a model defined by:
X features with shape N×L×V, and
Y label with shape N×V,
wherein N is the number of sentences and V is the number of different characters in the vocabulary such that for each row in X representing a sentence is predicted a row in Y representing a probability distribution of a next character.
14. The method of claim 1 , wherein generating synthetic phishing URLs using the generated phishing system, comprises:
defining a seed sentence using the identified patterns in the effective URLs that enable the effective URLs to avoid detection as phishing threats by the phishing detection system; and
predicting and adding a next character iteratively to the seed sentence according to the identified patterns.
15. The method of claim 14 , wherein generating synthetic phishing URLs using the generated phishing system, comprises:
upon generating the sentence in a full prose text, splitting the full prose text by http structure to produce a list of pseudo URLs; and
for each pseudo URL, concatenating a synthetic path of the pseudo URL to the domain compromised by the at least one effective phishing threat actor to produce a synthetic URL.
16. The method of claim 14 , further comprising generating variability in the prediction of the next character using a degeneration parameter.
17. The method of claim 15 , further comprising filtering the pseudo URLs by removing repeated pseudo URLs and dropping invalid characters in the pseudo URLs.
18. A non-transitory computer-readable storage medium comprising instructions that, when executed by a processing device, cause the processing device to:
identify a set of phishing threat actors, wherein each actor of the phishing threat actors is identified by identifying uniform resource locations (URLs) with certain path patterns on certain compromised domains;
determine an effectiveness rate of each actor of the set of phishing threat actors, wherein the effectiveness rate is a percentage of URLs produced by the actor that avoid detection as phishing threats by a phishing detection system;
select effective URLs from at least one effective phishing threat actor of the set of actors,
wherein the effective URLs are URLs that avoid detection as phishing threats by the phishing detection system, and
wherein the at least one effective phishing threat actor has an effectiveness rate exceeding a selected threshold;
generate, using a machine learning process, a phishing system using a long short-term memory (LSTM) network to identify patterns in the selected effective URLs that enable the selected effective URLs to avoid detection as phishing threats by the phishing detection system;
generate, by the machine learning process, a set of first artificial intelligence (AI) functions comprising a set of neurons that connect the machine learning process to a second machine learning process; and
generate, by the second machine learning process, a second AI function that classifies the effective URLs, wherein the second AI function is a sigmoid neuron;
generate synthetic phishing URLs using the generated phishing system and the identified patterns; and
train the phishing detection system according to the synthetic phishing URLs such that the phishing detection system is enhanced by the generated phishing system.
19. A computer system, comprising:
a processing device; and
memory in communication with the processing device and storing instructions that, when executed by the processing device, cause the processing device to:
generate, by a first machine learning process, a set of first artificial intelligence (AI) functions comprising a set of neurons that connect the machine learning process to a second machine learning process; and
generate, by the second machine learning process, a second AI function that classifies effective URLs, wherein the second AI function is a sigmoid neuron;
generate synthetic phishing uniform resource locations (URLs) based on the effective phishing URLs, by:
defining a seed sentence using identified patterns in the effective URLs that enable the effective URLs to avoid detection as phishing threats by a phishing detection system, wherein the patterns are identified by the machine learning process using a long short-term memory (LSTM) network;
predicting and adding a next character iteratively to the seed sentence according to the effective URLs;
producing a list of pseudo URLs from the sentence; and
for each pseudo URL, combining a synthetic path of the pseudo URL with a domain compromised by the at least one effective phishing threat actor to produce a synthetic URL; and
train the computer system according to the synthetic phishing URLs such that the computer system is enhanced by the training in that it can produce synthetic phishing URLs with increased effectiveness at avoiding detection as phishing threats by the phishing detection system.