System and method for cybersecurity reconnaissance, analysis, and score generation using distributed systems
A system and method for cybersecurity reconnaissance, analysis, and scoring that uses distributed, cloud-based computing services to provide sufficient scalability for analysis of enterprise IT networks using only publicly available characterizations. The system and method comprise an in-memory associative array which manages a queue of vulnerability search tasks through a public-facing proxy network. The public-facing proxy network has search nodes configurable to present the network to search tools in a desired manner to control certain aspects of the search to obtain the desired results. A distributed data processing engine and cloud-based storage are used to provide scalable computing power and storage. Each of the cloud-based computing services is containerized and orchestrated for management and efficient scaling purposes.
1. A system for cybersecurity reconnaissance, analysis, and scoring using distributed computing services, comprising:
a cloud computing platform comprising a hardware memory, a hardware processor, and a non-volatile storage device;
an in-memory associative array stored in the hardware memory;
a cloud-based storage bin stored on the non-volatile storage device;
a proxy server operating on the cloud computing platform configured to act as a public-facing proxy network, the public-facing proxy network comprising one or more selectable attribute nodes;
a user application comprising a first plurality of programming instructions stored in the hardware memory which, when operating on the hardware processor, causes the cloud computing platform to:
receive a domain name for reconnaissance and scoring; and
create a first queue of Internet search tasks for the domain name using an in-memory associative array service, the search tasks comprising searches for, and receipt of search results for, each of the following four types of domain name system records:
a domain name system search for domain name system records;
a domain name system search for domain name system sender policy framework records;
a domain name system search for domain name system domain-based message authentication, reporting, and conformance records; and
a domain name system search for domain name system zone transfer records;
implement the first queue of Internet search tasks through the one or more selectable attribute nodes of the public-facing proxy network;
identify Internet protocol addresses associated with the domain name from the domain name system records;
create a second queue of Internet protocol address scanning tasks for the identified Internet protocol addresses, the scanning tasks comprising an open port scan for each Internet protocol address identified;
implement the second queue of Internet protocol address scanning tasks and receive a list of open ports for the domain name; and
store the search results received from the first queue of Internet search tasks and the list of open ports from the second queue of Internet protocol address scanning tasks in the cloud-based storage bin; and
a distributed data processing engine comprising a second plurality of programming instructions stored in the hardware memory which, when operating on the hardware processor, causes the cloud computing platform to:
receive a cybersecurity scoring model, the cybersecurity scoring model comprising category weights for each of the following categories: domain name system records, the domain name system sender policy framework records, the domain name system domain-based message authentication, reporting, and conformance records, the zone transfer records, and the list of open ports, and further comprising an algorithm for combining the categories using the category weights;
retrieve the search results and the list of open ports stored in the cloud-based storage bin; and
calculate a cybersecurity score by applying the algorithm to the weighted categories; and
generate a cybersecurity profile for the domain name based on the cybersecurity score.
2. A method for cybersecurity reconnaissance, analysis, and scoring using distributed computing services, comprising the steps of:
using a user application operating on a cloud computing platform comprising a hardware memory, a hardware processor, and a non-volatile storage device, perform the steps of:
receiving a domain name for reconnaissance and scoring;
creating a first queue of Internet search tasks for the domain name using an in-memory associative array service operating on the cloud computing platform, the search tasks comprising searches for, and receipt of search results for, the following four types of domain name system records:
a domain name system search for domain name system records;
a domain name system search for domain name system sender policy framework records;
a domain name system search for domain name system domain-based message authentication, reporting, and conformance records; and
a domain name system search for domain name system zone transfer records;
implementing the first queue of Internet search tasks through one or more selectable attribute nodes of a public-facing proxy network operating on the cloud computing platform, the public-facing proxy network comprising a proxy server operating on the cloud computing platform configured to act as a public-facing proxy network and comprising one or more selectable attribute nodes;
identifying Internet protocol addresses associated with the domain name from the domain name system records;
creating a second queue of Internet protocol address scanning tasks for the identified Internet protocol addresses, the scanning tasks comprising an open port scan for each Internet protocol address identified;
implementing the second queue of Internet protocol address scanning tasks through the one or more selectable attribute nodes of the public-facing proxy network and receiving a list of open ports for the domain name; and
storing the search results received from the first queue of Internet search tasks and the list of open ports from the second queue of Internet protocol address scanning tasks in a cloud-based storage bin located on the non-volatile storage device; and
using a distributed data processing engine operating on the cloud computing platform, perform the steps of:
receiving a cybersecurity scoring model, the cybersecurity scoring model comprising category weights for each of the following categories: domain name system records, the domain name system sender policy framework records, the domain name system domain-based message authentication, reporting, and conformance records, the zone transfer records, and the list of open ports, and further comprising an algorithm for combining the categories using the category weights;
retrieving the search results and the list of open ports stored in the cloud-based storage bin; and
calculating a cybersecurity score by applying the algorithm to the weighted categories; and
generate a cybersecurity profile for the domain name based on the cybersecurity score.