Systems and methods for in-process URL condemnation
A universal resource locator (URL) collider processes a click event referencing a URL and directs a browser to a page at the URL. While the page is being rendered by the browser with page data from a web server, the URL collider intercepts the page data including events associated with rendering the page, determines microfeatures of the page such as Document Object Model objects and any URLs referenced by the page, applies detection rules, tags as evidence any detected bad microfeature, bad URL, or suspicious sequence of events, and stores the evidence in an evidence database. Based on the evidence, a judge module dynamically determines whether to condemn the URL before or just in time as the page at the URL is fully rendered by the browser. If so, the browser is directed to a safe location or a notification page.
1. A method for in-process universal resource locator (URL) condemnation, the method comprising:
directing, by the computer system, a browser to a page referenced by a URL of a click event, wherein the page is hosted by a server operating in a computing network; and
while the page is being rendered by the browser with page data from the server, performing, by the computer system:
determining, from the page data, microfeatures of the page and any URLs referenced by the page, wherein the microfeatures comprise Document Object Model (DOM) objects;
applying detection rules to events associated with rendering the page in the browser, the microfeatures of the page, and the URLs referenced by the page, wherein the detection rules comprise a rule identifying a microfeature as bad, a rule identifying a URL as bad, or a rule identifying a sequence of events as suspicious;
tagging as evidence any detected bad microfeature, any detected bad URL, and any detected suspicious sequence of events; and
storing the evidence in a database, wherein a subsequent computing facility is operable to dynamically determine, based on the evidence in the database, whether to condemn the URL before or just in time as the page of the URL is fully rendered by the browser.
2. The method according to claim 1 , wherein the click event referencing the URL is obtained or received from a processing queue, a data store, a URL feed, a service which handles URL threats, an agent of the computer system, or an email server communicatively connected to the computer system.
3. The method according to claim 1 , wherein the browser comprises a headless browser.
4. The method according to claim 3 , wherein the click event is captured by the headless browser when a user clicks on a URL embedded in a document.
5. The method according to claim 1 , further comprising:
prior to directing the browser to render the page linked by the URL, querying the subsequent computing facility on whether the URL in the click event is known to the subsequent computing facility, wherein the subsequent computing facility compares the URL in the click event with known URLs stored in a local cache or the evidence database and returns, prior to the page at the URL being fully rendered by the browser, a verdict indicating whether the URL in the click event is to be condemned.
6. The method according to claim 1 , wherein the rule identifying the suspicious sequence of events includes detection of a loading of an inline frame by the browser.
7. A system for in-process universal resource locator (URL) condemnation, the system comprising:
a processor;
a non-transitory computer-readable medium; and
instructions stored on the non-transitory computer-readable medium and translatable by the processor for:
directing a browser to a page referenced by a URL of a click event, wherein the page is hosted by a server operating in a computing network; and
while the page is being rendered by the browser with page data from the server, performing:
determining, from the page data, microfeatures of the page and any URLs referenced by the page, wherein the microfeatures comprise Document Object Model (DOM) objects;
applying detection rules to events associated with rendering the page in the browser, the microfeatures of the page, and the URLs referenced by the page, wherein the detection rules comprise a rule identifying a microfeature as bad, a rule identifying a URL as bad, or a rule identifying a sequence of events as suspicious;
tagging as evidence any detected bad microfeature, any detected bad URL, and any detected suspicious sequence of events; and
storing the evidence in a database, wherein a subsequent computing facility is operable to dynamically determine, based on the evidence in the database, whether to condemn the URL before or just in time as the page of the URL is fully rendered by the browser.
8. The system of claim 7 , wherein the click event referencing the URL is obtained or received from a processing queue, a data store, a URL feed, a service which handles URL threats, an agent of the system, or an email server communicatively connected to the system.
9. The system of claim 7 , wherein the browser comprises a headless browser.
10. The system of claim 9 , wherein the click event is captured by the headless browser when a user clicks on a URL embedded in a document.
11. The system of claim 7 , further comprising:
prior to directing the browser to render the page linked by the URL, querying the subsequent computing facility on whether the URL in the click event is known to the subsequent computing facility, wherein the subsequent computing facility compares the URL in the click event with known URLs stored in a local cache or the evidence database and returns, prior to the page at the URL being fully rendered by the browser, a verdict indicating whether the URL in the click event is to be condemned.
12. The system of claim 7 , wherein the rule identifying the suspicious sequence of events includes detection of a loading of an inline frame by the browser.
13. A computer program product for in-process universal resource locator (URL) condemnation, the computer program product comprising a non-transitory computer-readable medium storing instructions translatable by a processor of a computer system for:
directing a browser to a page referenced by a URL of a click event, wherein the page is hosted by a server operating in a computing network; and
while the page is being rendered by the browser with page data from the server, performing:
determining, from the page data, microfeatures of the page and any URLs referenced by the page, wherein the microfeatures comprise Document Object Model (DOM) objects;
applying detection rules to events associated with rendering the page in the browser, the microfeatures of the page, and the URLs referenced by the page, wherein the detection rules comprise a rule identifying a microfeature as bad, a rule identifying a URL as bad, or a rule identifying a sequence of events as suspicious;
tagging as evidence any detected bad microfeature, any detected bad URL, and any detected suspicious sequence of events; and
storing the evidence in a database, wherein a subsequent computing facility is operable to dynamically determine, based on the evidence in the database, whether to condemn the URL before or just in time as the page of the URL is fully rendered by the browser.
14. The computer program product of claim 13 , wherein the click event referencing the URL is obtained or received from a processing queue, a data store, a URL feed, a service which handles URL threats, an agent of the computer system, or an email server communicatively connected to the computer system.
15. The computer program product of claim 13 , wherein the browser comprises a headless browser running on the device.
16. The computer program product of claim 15 , wherein the click event is captured by the headless browser when a user clicks on a URL embedded in a document.
17. The computer program product of claim 13 , wherein the rule identifying the suspicious sequence of events includes detection of a loading of an inline frame by the browser.