Modular network scan orchestration
An example of the modular outpost pipeline can make complex branching decisions at stages along the pipeline based on attributes of the data already gathered. This allows modifications to scans in progress within the outpost-based pipeline, and near-real-time sharing of specific scan data to external tools. The logic necessary can be implemented in any outpost pipeline stage but is preferentially accomplished by using secondary orchestrator stages inserted after the pipeline stages where relevant data has been acquired.
1 . A modular outpost system comprising:
a server; and
a memory with instructions which, when executed by a processor, cause the system to create a modular outpost engine having:
a primary orchestrator module configured to generate an initial pipeline for a scan or set of scans, wherein the initial pipeline is based upon a static target list of addresses;
a plurality of secondary orchestrator modules;
a plurality of scan stages; and
an avoid list that is used by the primary orchestrator module to avoid certain target addresses during network scanning, wherein the avoid list is hierarchical such that multiple avoid list entries for a single target address are combined to create a composite avoid entry;
wherein the primary orchestrator module is configured to randomize addresses across the static target list to minimize impact to bandwidth within contiguous network segments, minimize load on network devices, and reduce false alarms in Intrusion Detection systems;
wherein the primary orchestrator module conducts a covering search for a target address in the static target list against the avoid list that returns nodes that cover the target address;
wherein avoid list entries from the returned nodes are subdivided by scan stage and merged through delegation; and
wherein the plurality of secondary orchestrator modules is associated with the plurality of scan stages and dynamically defines a modular outpost pipeline.
2 . The modular outpost system of claim 1 , wherein the initial pipeline is further based upon one or more of the group consisting of: ranges and Classless Inter-Domain Routing-formatted addresses.
3 . The modular outpost system of claim 1 , wherein the modular outpost pipeline is a single pipeline.
4 . The modular outpost system of claim 1 , wherein the modular outpost pipeline is a branching pipeline.
5 . The modular outpost system of claim 1 , wherein at least some of the plurality of secondary orchestrator modules and the plurality of scan stages are configured to publish specific attributes from scan results of hierarchical topics.
6 . The modular outpost system of claim 5 , wherein at least some of the plurality of secondary orchestrator modules and the plurality of scan stages are subscribers to the specific attributes from the scan results of hierarchical topics.
7 . The modular outpost system of claim 1 , further comprising a user device coupled to the server and the modular outpost engine via a network.
8 . The modular outpost system of claim 7 , wherein the user device is usable by an operator to set or modify the modular outpost pipeline.
9 . The modular outpost system of claim 1 , wherein the avoid list comprises a list of targets that react adversely to being scanned and can be configured to require wholesale avoidance of a given target or stage, selective avoidance of some stages, configuration of a single address, or configuration of multiple addresses.
10 . The modular outpost system of claim 9 , wherein the avoid list is modular such that adding new stages to the plurality of scan stages does not require changes to the primary orchestrator module, the plurality of secondary orchestrator modules, or existing ones of the plurality of scan stages.
11 . A method comprising:
assigning a pipeline for scanning a plurality of targets at a primary orchestrator module based upon a static target list of IP addresses;
generating an avoid list that is used by the primary orchestrator module to avoid certain target addresses during network scanning, wherein the avoid list is hierarchical such that multiple avoid list entries for a single target address are combined to create a composite avoid entry;
scanning the plurality of targets at a plurality of stages as defined by the pipeline, wherein the primary orchestrator module is configured to randomize addresses across the static target list to minimize impact to bandwidth within contiguous network segments, minimize load on network devices, and reduce false alarms in Intrusion Detection systems, wherein the primary orchestrator module conducts a covering search for a target address in the static target list against the avoid list that returns nodes that cover the target address, and wherein avoid list entries from the returned nodes are subdivided by scan stage and merged through delegation; and
modifying the pipeline at one of a plurality of secondary orchestrator modules.
12 . The method of claim 11 , wherein modifying the pipeline comprises updating a single pipeline.
13 . The method of claim 11 , wherein modifying the pipeline comprises updating a branching pipeline.
14 . The method of claim 11 , further comprising:
publishing, by at least some of the plurality of secondary orchestrator modules and the plurality of stages, specific attributes from scan results of hierarchical topics; and
obtaining the specific attributes from the scan results of hierarchical topics by others of the plurality of secondary orchestrator modules and the plurality of stages.
15 . The method of claim 11 , wherein the avoid list comprises a list of targets that react adversely to being scanned and can be configured to require wholesale avoidance of a given target or stage, selective avoidance of some stages, configuration of a single address, or configuration of multiple addresses.
16 . The method of claim 11 , wherein the avoid list is modular such that adding new stages to the plurality of stages does not require changes to the primary orchestrator module, the plurality of secondary orchestrator modules, or existing ones of the plurality of stages.