Resolving redirects for enhanced security
Implementations include receiving, by a redirection resolver, a URL identifying a location of a network resource, processing, by the redirection resolver, the URL to provide a set of results including a set of redirection URLs, the set of redirection URLs including one or more redirections between the URL and an end URL, processing the set of redirection URLs to provide input to a machine learning (ML) model that generates an output based on the set of redirection URLs, determining an error value associated with the URL, and providing an indicator assigned to the URL based on the error value, the indicator indicating whether the URL is determined to be potentially malicious.
1. A computer-implemented method for identifying potentially malicious uniform resource locator (URLs) based on redirection behavior, the method being executed by one or more processors and comprising:
receiving, by a redirection resolver, a URL identifying a location of a network resource;
processing, by the redirection resolver, the URL to provide a set of results comprising a set of redirection URLs, the set of redirection URLs comprising one or more redirections between the URL and an end URL;
processing the set of redirection URLs to provide input to a machine learning (ML) model that generates an output based on the set of redirection URLs;
determining an error value associated with the URL; and
providing an indicator assigned to the URL based on the error value, the indicator indicating whether the URL is determined to be potentially malicious.
2. The method of claim 1 , wherein processing the URL to provide the set of results comprises processing the URL through a hypertext transfer protocol (HTTP) scanner based on a set of HTTP methods to provide a sub-set of results that represents one or more server redirects of the URL.
3. The method of claim 2 , wherein processing the URL through the HTTP scanner is executed multiple times to account for any inconsistency in redirections.
4. The method of claim 2 , wherein processing the URL through the HTTP scanner comprises processing both HTTP and HTTP secure (HTTPS) protocols.
5. The method of claim 1 , wherein processing the URL to provide the set of results comprises processing the URL through a browser simulator to provide a sub-set of results that represents one or more page redirects of the URL.
6. The method of claim 5 , wherein processing the URL through the browser simulator is executed multiple times to account for any inconsistency in redirections.
7. The method of claim 5 , wherein processing the URL through the browser simulator comprises processing both HTTP and HTTP secure (HTTPS) protocols.
8. The method of claim 1 , wherein the ML model comprises an autoencoder that is trained based on redirection behavior of a set of trusted URLs.
9. The method of claim 8 , wherein the autoencoder comprises an embedding layer, an encoding layer, an encoded URL layer, and a decoding layer.
10. The method of claim 1 , wherein providing the indicator comprises:
comparing the error value to a threshold error value; and
determining that the error value exceeds the threshold error value, and in response, setting the indicator to a value that indicates that the URL is potentially malicious.
11. The method of claim 1 , wherein providing the indicator comprises:
comparing the error value to a threshold error value; and
determining that the error value does not exceed the threshold error value, and in response, setting the indicator to a value that indicates that the URL is not potentially malicious.
12. A non-transitory computer-readable storage medium coupled to one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations for identifying potentially malicious uniform resource locator (URLs) based on redirection behavior, the operations comprising:
receiving, by a redirection resolver, a URL identifying a location of a network resource;
processing, by the redirection resolver, the URL to provide a set of results comprising a set of redirection URLs, the set of redirection URLs comprising one or more redirections between the URL and an end URL;
processing the set of redirection URLs to provide input to a machine learning (ML) model that generates an output based on the set of redirection URLs;
determining an error value associated with the URL; and
providing an indicator assigned to the URL based on the error value, the indicator indicating whether the URL is determined to be potentially malicious.
13. The computer-readable storage medium of claim 12 , wherein processing the URL to provide the set of results comprises processing the URL through a hypertext transfer protocol (HTTP) scanner based on a set of HTTP methods to provide a sub-set of results that represents one or more server redirects of the URL.
14. The computer-readable storage medium of claim 13 , wherein processing the URL through the HTTP scanner is executed multiple times to account for any inconsistency in redirections.
15. The computer-readable storage medium of claim 13 , wherein processing the URL through the HTTP scanner comprises processing both HTTP and HTTP secure (HTTPS) protocols.
16. The computer-readable storage medium of claim 12 , wherein processing the URL to provide the set of results comprises processing the URL through a browser simulator to provide a sub-set of results that represents one or more page redirects of the URL.
17. The computer-readable storage medium of claim 16 , wherein processing the URL through the browser simulator is executed multiple times to account for any inconsistency in redirections.
18. The computer-readable storage medium of claim 16 , wherein processing the URL through the browser simulator comprises processing both HTTP and HTTP secure (HTTPS) protocols.
19. The computer-readable storage medium of claim 12 , wherein the ML model comprises an autoencoder that is trained based on redirection behavior of a set of trusted URLs.
20. The computer-readable storage medium of claim 19 , wherein the autoencoder comprises an embedding layer, an encoding layer, an encoded URL layer, and a decoding layer.
21. The computer-readable storage medium of claim 12 , wherein providing the indicator comprises:
comparing the error value to a threshold error value; and
determining that the error value exceeds the threshold error value, and in response, setting the indicator to a value that indicates that the URL is potentially malicious.
22. The computer-readable storage medium of claim 12 , wherein providing the indicator comprises:
comparing the error value to a threshold error value; and
determining that the error value does not exceed the threshold error value, and in response, setting the indicator to a value that indicates that the URL is not potentially malicious.
23. A system, comprising:
one or more processors; and
a computer-readable storage device coupled to the one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations for identifying potentially malicious uniform resource locator (URLs) based on redirection behavior, the operations comprising:
receiving, by a redirection resolver, a URL identifying a location of a network resource;
processing, by the redirection resolver, the URL to provide a set of results comprising a set of redirection URLs, the set of redirection URLs comprising one or more redirections between the URL and an end URL;
processing the set of redirection URLs to provide input to a machine learning (ML) model that generates an output based on the set of redirection URLs;
determining an error value associated with the URL; and
providing an indicator assigned to the URL based on the error value, the indicator indicating whether the URL is determined to be potentially malicious.
24. The system of claim 23 , wherein processing the URL to provide the set of results comprises processing the URL through a hypertext transfer protocol (HTTP) scanner based on a set of HTTP methods to provide a sub-set of results that represents one or more server redirects of the URL.
25. The system of claim 24 , wherein processing the URL through the HTTP scanner is executed multiple times to account for any inconsistency in redirections.
26. The system of claim 24 , wherein processing the URL through the HTTP scanner comprises processing both HTTP and HTTP secure (HTTPS) protocols.
27. The system of claim 23 , wherein processing the URL to provide the set of results comprises processing the URL through a browser simulator to provide a sub-set of results that represents one or more page redirects of the URL.
28. The system of claim 27 , wherein processing the URL through the browser simulator is executed multiple times to account for any inconsistency in redirections.
29. The system of claim 27 , wherein processing the URL through the browser simulator comprises processing both HTTP and HTTP secure (HTTPS) protocols.
30. The system of claim 23 , wherein the ML model comprises an autoencoder that is trained based on redirection behavior of a set of trusted URLs.
31. The system of claim 30 , wherein the autoencoder comprises an embedding layer, an encoding layer, an encoded URL layer, and a decoding layer.
32. The system of claim 23 , wherein providing the indicator comprises:
comparing the error value to a threshold error value; and
determining that the error value exceeds the threshold error value, and in response, setting the indicator to a value that indicates that the URL is potentially malicious.
33. The system of claim 23 , wherein providing the indicator comprises:
comparing the error value to a threshold error value; and
determining that the error value does not exceed the threshold error value, and in response, setting the indicator to a value that indicates that the URL is not potentially malicious.