Rating organization cybersecurity using active and passive external reconnaissance
A system for cybersecurity rating using active and passive external reconnaissance, that uses a web crawler that sends message prompts to external hosts and receives responses from external hosts, a time-series data store that produces time-series data from the message responses, and a directed computational graph module that analyzes the time-series data to produce a weighted score representing the overall cybersecurity state of an organization.
1. An advanced cyber decision platform for external network reconnaissance and cybersecurity rating, the platform comprising:
a computing device comprising a memory and a processor;
a time-series data module comprising a first plurality of programming instructions stored in the memory of, and operating on the processor of, the computing device, wherein the first plurality of programming instructions, when operating on the processor, cause the computing device to:
receive leak information and reconnaissance data about a company from a web crawler; and
produce time-series data based on at least a portion of the leak information and reconnaissance data;
a directed computational graph module comprising a second plurality of programming instructions stored in the memory of, and operating on the processor of, the computing device, wherein the second plurality of programming instructions, when operating on the processor, cause the computing device to:
retrieve at least a portion of the stored time series data;
produce a directed computational graph by performing a plurality of transformation operations on at least a portion of the time-series data, wherein:
each transformation operation sends a message output to subsequent transformation operations;
the directed computational graph comprises nodes and edges, the nodes representing the transformation operations and the edges representing message outputs between the nodes; and
one or more of the transformation operations are linearization of non-linear operations that are created when they are ready to be computed; and
produce a weighted score based on at least a portion of the transformation operations, wherein the weighted score represents a risk of external cyber-security threats to the company based on the collected leak information and the reconnaissance data; and
a web crawler comprising at least a processor, a memory, and a plurality of programming instructions stored in the memory and operating on the processor, wherein the programmable instructions, when operating on the processor, cause the processor to:
collect leak information about a company from publicly-available records using domain name system entries and internet protocol addresses associated with the company;
use the collected leak information to obtain reconnaissance data for websites and Internet applications used by, or affecting, the company, the reconnaissance data comprising:
identifying portions each website or application that web crawlers are requested to ignore;
fingerprinting each website or application to identify characteristic patterns or markers that may be used to identify host or application details;
checking each website's administrative pages to determine if any administrative portals are vulnerable to cyber-attacks; and
determining a patching frequency of each website or application; and
provide the collected leak information and the reconnaissance data to the time-series data module.
2. A method for external network reconnaissance and cybersecurity rating, comprising the steps of:
Using a web crawler to:
collect leak information about a company from publicly-available records using domain name system entries and internet protocol addresses associated with the company;
use the collected leak information to obtain reconnaissance data for websites and Internet applications used by, or affecting, the company, the reconnaissance data comprising:
identifying portions each website or application that web crawlers are requested to ignore;
fingerprinting each website or application to identify characteristic patterns or markers that may be used to identify host or application details;
checking each website's administrative pages to determine if any administrative portals are vulnerable to cyber-attacks; and
determining a patching frequency of each website or application; and
producing, using a time-series data module, time-series data based on at least a portion of the leak information and the reconnaissance data;
producing a directed computational graph by performing, using a directed computational graph module, a plurality of transformation operations on at least a portion of the time-series data, wherein:
each transformation operation sends a message output to subsequent transformation operations;
the directed computational graph comprises nodes and edges, the nodes representing the transformation operations and the edges representing message outputs between the nodes; and
one or more of the transformation operations are linearization of non-linear operations that are created when they are ready to be computed; and
producing a weighted score based on at least a portion of the transformation operations, wherein the weighted score represents a risk of external cyber-security threats to the company based on the collected leak information and the reconnaissance data.