Visual deep learning for inline phishing detection
Techniques for visual deep learning for inline phishing detection are disclosed. In some embodiments, a system/process/computer program product for visual deep learning for inline phishing detection includes extracting a logo from a screenshot of a web page; detecting phishing based on a match to at least one of a plurality of reference logos using a visual deep learning model and that a domain associated with the web page is not associated with an entity that matches the logo extracted from the web page; and performing a remedial action in response to determining that the web page is associated with phishing.
1 . A system, comprising:
a processor configured to:
extract a logo from a screenshot of a web page;
detect phishing based on a match to at least one of a plurality of reference logos using a visual deep learning model and that a domain associated with the web page is not associated with an entity that matches the logo extracted from the web page, comprising to:
input the extracted logo into the visual deep learning model to output a first feature vector and a second feature vector, wherein the visual deep learning model passes the extracted logo to at least two different sub-networks to generate the first feature vector and the second feature vector;
input a combination of the first feature vector and the second feature vector into a neural network to generate a logo embedding;
determine a similarity of the logo embedding associated with the logo and a reference logo embedding associated with the at least one reference logo; and
in response to a determination that the similarity fails to equal or exceed a similarity threshold, determine that the web page is associated with phishing; and
perform a remedial action in response to determining that the web page is associated with phishing; and
a memory coupled to the processor and configured to provide the processor with instructions.
2 . The system of claim 1 , wherein a set of candidate reference logos are embedded as feature vectors using the visual deep learning model and then clustered, filtered, and deduplicated to generate the plurality of reference logos.
3 . The system of claim 1 , wherein the remedial action includes injecting a warning in the web page, and wherein the detecting phishing is performed using a browser extension, browser plugin, enterprise browser, or a remote browser isolation (RBI) environment.
4 . The system of claim 1 , wherein the remedial action includes blocking the response.
5 . The system of claim 1 , wherein performing the remedial action includes adding a Uniform Resource Locator (URL) associated with a detected phishing server to a block list.
6 . The system of claim 1 , wherein a login form is detected in the screenshot of the web page.
7 . The system of claim 1 , wherein a Uniform Resource Locator (URL) is extracted from a request for the web page that was determined to be associated with phishing activity, and wherein the URL is added to a URL block list.
8 . The system of claim 1 , wherein an output of the visual deep learning model includes an embedding vector that is compared with a plurality of embedding vectors for the plurality of reference logos.
9 . The system of claim 1 , wherein the plurality of reference logos includes a plurality of embedded reference logos, and wherein a new reference logo is added as a new embedded reference logo for potentially matching with the extracted logo without retraining of the visual deep learning model.
10 . The system of claim 1 , wherein performing the remedial action includes providing an explanation of the phishing detection that includes a matching reference logo and information indicating that the web page is not associated with an entity that matches the logo extracted from the web page.
11 . A method, comprising:
extracting a logo from a screenshot of a web page;
detecting phishing based on a match to at least one of a plurality of reference logos using a visual deep learning model and that a domain associated with the web page is not associated with an entity that matches the logo extracted from the web page, comprising:
inputting the extracted logo into the visual deep learning model to output a first feature vector and a second feature vector, wherein the visual deep learning model passes the extracted logo to at least two different sub-networks to generate the first feature vector and the second feature vector;
inputting a combination of the first feature vector and the second feature vector into a neural network to generate a logo embedding;
determining a similarity of the logo embedding associated with the logo and a reference logo embedding associated with the at least one reference logo; and
in response to a determination that the similarity fails to equal or exceed a similarity threshold, determining that the web page is associated with phishing; and
performing a remedial action in response to determining that the web page is associated with phishing.
12 . The method of claim 11 , wherein a set of candidate reference logos are embedded as feature vectors using the visual deep learning model and then clustered, filtered, and deduplicated to generate the plurality of reference logos.
13 . The method of claim 11 , wherein the remedial action includes injecting a warning in the web page, and wherein the detecting phishing is performed using a browser extension, browser plugin, enterprise browser, or a remote browser isolation (RBI) environment.
14 . The method of claim 11 , wherein the remedial action includes blocking the response.
15 . The method of claim 11 , wherein performing the remedial action includes adding a Uniform Resource Locator (URL) associated with a detected phishing server to a block list.
16 . The method of claim 11 , wherein a Uniform Resource Locator (URL) is extracted from a request for the web page that was determined to be associated with phishing activity, and wherein the URL is added to a URL block list.
17 . A computer program product embodied in a non-transitory computer readable medium and comprising computer instructions for:
extracting a logo from a screenshot of a web page;
detecting phishing based on a match to at least one of a plurality of reference logos using a visual deep learning model and that a domain associated with the web page is not associated with an entity that matches the logo extracted from the web page, comprising:
inputting the extracted logo into the visual deep learning model to output a first feature vector and a second feature vector, wherein the visual deep learning model passes the extracted logo to at least two different sub-networks to generate the first feature vector and the second feature vector;
inputting a combination of the first feature vector and the second feature vector into a neural network to generate a logo embedding;
determining a similarity of the logo embedding associated with the logo and a reference logo embedding associated with the at least one reference logo; and
in response to a determination that the similarity fails to equal or exceed a similarity threshold, determining that the web page is associated with phishing; and
performing a remedial action in response to determining that the web page is associated with phishing.
18 . The computer program product of claim 17 , wherein a set of candidate reference logos are embedded as feature vectors using the visual deep learning model and then clustered, filtered, and deduplicated to generate the plurality of reference logos.
19 . The computer program product of claim 17 , wherein the remedial action includes injecting a warning in the web page, and wherein the detecting phishing is performed using a browser extension, browser plugin, enterprise browser, or a remote browser isolation (RBI) environment.
20 . The computer program product of claim 17 , wherein the remedial action includes blocking the response.