Rating organization cybersecurity using active and passive external reconnaissance
A system for cybersecurity rating using active and passive external reconnaissance, that uses a web crawler that sends message prompts to external hosts and receives responses from external hosts, a time-series data store that produces time-series data from the message responses, and a directed computational graph module that analyzes the time-series data to produce a weighted score representing the overall cybersecurity state of an organization.
1. An advanced cyber decision platform for external network reconnaissance and cybersecurity rating, the platform comprising:
a computing device comprising a hardware memory and a processor;
a time-series data module comprising a first plurality of programming instructions stored in the hardware memory of, and operating on the processor of, the computing device, wherein the first plurality of programming instructions, when operating on the processor, cause the computing device to:
receive leak information and reconnaissance data about a company from a web crawler;
produce time-series data based on at least a portion of the leak information and reconnaissance data;
a directed computational graph module comprising a second plurality of programming instructions stored in the hardware memory of, and operating on the processor of, the computing device, wherein the second plurality of programming instructions, when operating on the processor, cause the computing device to:
retrieve at least a portion of the time-series data;
receive a directed computational graph representing a workflow for performing a plurality of transformation operations on at least a portion of the time-series data, wherein:
the directed computational graph comprises nodes and edges, the nodes representing the transformation operations and the edges representing message outputs between the nodes; and
each transformation operation sends a message output to subsequent transformation operations; and
produce a weighted score based on at least a portion of the transformation operations, wherein the weighted score represents a risk of external cyber-security threats to the company based on the collected leak information and the reconnaissance data; and
a web crawler comprising a third plurality of programming instructions stored in the hardware memory of, and operating on the processor of, the computing device, wherein the third plurality of programming instructions, when operating on the processor, cause the computing device to:
collect leak information about the company from publicly-available records using domain name system entries and internet protocol addresses associated with the company;
use the collected leak information to obtain reconnaissance data for websites and Internet applications used by, or affecting, the company, the reconnaissance data comprising:
identifying portions of each website or application that web crawlers are requested to ignore;
fingerprinting each website or application to identify characteristic patterns or markers that may be used to identify host or application details; and
checking each website's administrative pages to determine if any administrative portals are vulnerable to cyber-attacks; and
provide the collected leak information and the reconnaissance data to the time-series data module.
2. A method for external network reconnaissance and cybersecurity rating, comprising the steps of:
using a web crawler to:
collect leak information about a company from publicly-available records using domain name system entries and internet protocol addresses associated with the company;
use the collected leak information to obtain reconnaissance data for websites and Internet applications used by, or affecting, the company, the reconnaissance data comprising:
identifying portions each website or application that web crawlers are requested to ignore;
fingerprinting each website or application to identify characteristic patterns or markers that may be used to identify host or application details; and
checking each website's administrative pages to determine if any administrative portals are vulnerable to cyber-attacks;
producing, using a time-series data module, time-series data based on at least a portion of the collected leak information and the reconnaissance data;
receive a directed computational graph representing a workflow for performing a plurality of transformation operations on at least a portion of the time-series data, wherein:
the directed computational graph comprises nodes and edges, the nodes representing the transformation operations and the edges representing message outputs between the nodes; and
each transformation operation sends a message output to subsequent transformation operations; and
producing a weighted score based on at least a portion of the transformation operations, wherein the weighted score represents a risk of external cyber-security threats to the company based on the collected leak information and the reconnaissance data.