System and method for improved end-to-end cybersecurity machine learning and deployment
The presently disclosed subject matter includes an apparatus that receives a dataset with values associated with different digital resources captured from a group of compute devices. The apparatus includes a feature extractor, to generate a set of feature vectors, each feature vector from the set of feature vectors associated with a set of data included in the received dataset. The apparatus uses the set of feature vectors to validate multiple machine learning models trained to determine whether a digital resource is associated with a cyberattack. The apparatus selects at least one active machine learning model and sets the remaining trained machine learning models to operate in an inactive mode. The active machine learning model generates a signal to alert a security administrator, blocks a digital resource from loading at a compute device, or executes other remedial action, upon a determination that the digital resource is associated with a cyberattack.
1. A method for improving a cybersecurity detection system using a plurality of machine learning models, comprising:
generating a plurality of trained machine learning models, each trained machine learning model from the plurality of trained machine learning models trained to classify whether a digital resource from a first plurality of digital resources is associated with any cyberattack of a plurality of cyberattacks;
selecting, based on at least one performance metric, a first machine learning model from the plurality of trained machine learning models to operate in an in-line mode and a different second machine learning model from the plurality of trained machine learning models to operate in an out-of-band mode,
the in-line mode being characterized by classifying a second plurality of digital resources to generate classifications and using the classifications in determining remedial actions including alerts,
the out-of-band mode being characterized by classifying the second plurality of digital resources to generate classifications without using the classifications in determining a remedial action including alerts even when the classifications indicate an association with a cyberattack of the plurality of cyberattacks;
deploying, to one or more compute devices via a network, both the first machine learning model and the second machine learning model to classify a third plurality of digital resources;
evaluating performance following execution of the first machine learning model and the second machine learning model on the one or more compute devices;
sending a signal to the one or more compute devices to change, based on the performance, a configuration of the first machine learning model and a configuration of the second machine learning model between the in-line mode and the out-of-band mode; and
deploying, to the one or more compute devices via the network, a third machine learning model different from the first machine learning model and the second machine learning model from the plurality of trained machine learning models to operate in the in-line mode when the first machine learning model and the second machine learning model fail to satisfy a performance threshold.
2. The method of claim 1 , wherein the deploying of the third machine learning model causes the one or more compute devices to initiate the remedial action when a statistically-combined classification value of a classification using the first machine learning model and a classification using the third machine learning model indicates a digital resource from the third plurality of digital resources is associated with a cyberattack from the plurality of cyberattacks.
3. The method of claim 1 , wherein: when in the out-of-band mode, the second machine learning model classifies the second plurality of digital resources based on copies of the second plurality of digital resources.
4. The method of claim 1 , further comprising: receiving, based on an event-based condition, a labeled dataset; and
retraining the plurality of trained machine learning models based on the labeled dataset.
5. The method of claim 4 , wherein the event-based condition is met when a number of feature values in a set of feature values received from a set of data sources reaches a predetermined threshold and the set of feature values have not been used to train, validate, or evaluate the plurality of trained machine learning models.
6. The method of claim 4 , wherein the event-based condition is met when a predetermined threshold number of new types of malware has been detected via one or more data sources.
7. The method of claim 1 , wherein satisfy the performance threshold includes at least one of (i) achieving a performance above an accuracy threshold, or (ii) achieving a performance below an error threshold.
8. The method of claim 1 , wherein:
the deploying the third machine learning model includes selecting the third machine learning model from the plurality of trained machine learning models;
the deploying the third machine learning model causes the one or more compute devices to initiate the remedial action when the third machine learning model classifies a digital resource from the third plurality of digital resources as associated with a cyberattack from the plurality of cyberattacks.
9. The method of claim 1 , wherein the one or more compute devices include a plurality of compute devices.
10. The method of claim 1 , further comprising:
executing the third machine model on a plurality of compute devices including the one or more compute devices to improve cybersecurity in a production network in response to the evaluation of performance.
11. A system for improving cybersecurity detection, comprising:
at least one processor; and
at least one memory storing instructions which, when executed by the at least one processor, causes the at least one processor to:
receive, over a network, at least a subset of machine learning models from a set of machine learning models;
deploy the subset of machine learning models configured to be switched, in response to a received switching signal over the network, between an in-line mode and an out-of-band mode to classify a plurality of digital resources as associated with any cyberattack from a plurality of cyberattacks,
wherein the deployed subset of machine learning models includes both a first machine learning model to operate in the in-line mode and a different second machine learning model to operate in the out-of-band mode, wherein the received switching signal is based on evaluating performance of the first machine learning model and the second machine learning model,
the in-line mode being characterized by classifying digital resources to generate classifications and using the classifications in determining remedial actions including alerts, and
the out-of-band mode being characterized by classifying the digital resources to generate classifications without using the classification in determining a remedial action including alerts even when the classifications indicate an association with a cyberattack from the plurality of cyberattacks; and
execute the subset of machine learning models on a plurality of received digital resources;
initiate one or more remedial actions in response to at least one machine learning model from the subset of machine learning models configured in the in-line mode classifying a digital resource of the plurality of received digital resources as associated with a cyberattack from the plurality of cyberattacks; and
deploy a third machine learning model different from the first machine learning model and the second machine learning model from a plurality of trained machine learning models to operate in the in-line mode when the first machine learning model and the second machine learning model fail to satisfy a performance threshold.
12. The system of claim 11 , wherein:
the least one memory further stores instructions which, when executed by the at least one processor, causes the at least one processor to:
in response to the deploying of the third machine learning model, initiate the one or more remedial actions when a statistically-combined classification value of a classification using the first machine learning model and a classification using the third machine learning model indicates a digital resource from the plurality of digital resources is associated with a cyberattack from the plurality of cyberattacks.
13. The system of claim 11 , wherein satisfy the performance threshold includes at least one of (i) achieving a performance above an accuracy threshold, or (ii) achieving a performance below an error threshold.
14. The system of claim 11 , wherein: when in the out-of-band mode, the second machine learning model classifies the plurality of digital resources based on copies of the plurality of digital resources.
15. A computing device with improved cybersecurity detection using a plurality of machine learning models, comprising:
at least one processor; and
at least one memory storing instructions which, when executed by the at least one processor, causes the at least one processor to:
deploy a first machine learning model and a second machine learning model different from the first machine learning model to classify each digital resource from a plurality of digital resources;
the first machine learning model to operate in an in-line mode and the second machine learning model to operate in an out-of-band mode,
the in-line mode being characterized by classifying digital resources to generate classifications and using the classifications in determining remedial actions including alerts, and
the out-of-band mode being characterized by classifying the digital resources to generate classifications without using the classifications in determining remedial actions including alerts even when the classifications indicate an association with a cyberattack from a plurality of cyberattacks;
execute the first machine learning model and the second machine learning model on each digital resource from the plurality of digital resources or copies thereof;
initiate one or more remedial actions in response to the first machine learning model configured in the in-line mode classifying a first digital resource of the plurality of digital resources as associated with a cyberattack from the plurality of cyberattacks;
receive a switching signal over the network in response to an evaluation of performance of the first machine learning model failing to satisfy at least one performance threshold, the switching signal causing (i) the first machine learning model to operate in the out-of-band mode, and ii) the second machine learning model to operate in the in-line mode, when classifying a second digital resource; and
deploy a third machine learning model different from the first machine learning model and the second machine learning model from a plurality of trained machine learning models to operate in the in-line mode when the first machine learning model and the second machine learning model fail to satisfy a performance threshold.
16. The computing device of claim 15 , wherein:
the at least one memory storing instructions which causes the at least one processor to:
maintain the first machine learning mode in the in-line mode and the second machine learning mode in the out-of-band mode until receiving a signal to switch between the in-line mode to the out-of-band mode.
17. The computing device of claim 15 , wherein:
the least one memory further stores instructions which, when executed by the at least one processor, causes the at least one processor to:
in response to the deploying the third machine learning model, initiate the one or more remedial action when a statistically-combined classification value of a classification using the first machine learning model and a classification using the third machine learning model indicates a digital resource from the plurality of digital resources is associated with a cyberattack from the plurality of cyberattacks.
18. The computing device of claim 15 , wherein satisfy the performance threshold includes at least one of (i) achieving a performance above an accuracy threshold, or (ii) achieving a performance below an error threshold.
19. The computing device of claim 15 , wherein: when in the out-of-band mode, the second machine learning model classifies the digital resources based on copies of the digital resources.
20. A method for improving a cybersecurity detection system using a plurality of machine learning models, comprising:
generating a plurality of trained machine learning models, each trained machine learning model from the plurality of trained machine learning models trained to classify whether a digital resource from a first plurality of digital resources is associated with any cyberattack of a plurality of cyberattacks;
selecting, based on at least one performance metric, a first machine learning model from the plurality of trained machine learning models to operate in an in-line mode and a different second machine learning model from the plurality of trained machine learning models to operate in an out-of-band mode,
the in-line mode being characterized by classifying a second plurality of digital resources to generate classifications and using the classifications in determining remedial actions including alerts,
the out-of-band mode being characterized by classifying the second plurality of digital resources to generate classifications without using the classifications in determining a remedial action including alerts even when the classifications indicate an association with a cyberattack of the plurality of cyberattacks;
deploying, to one or more compute devices via a network, both the first machine learning model and the second machine learning model to classify a third plurality of digital resources;
evaluating performance following execution of the first machine learning model and the second machine learning model on the one or more compute devices;
sending a signal to the one or more compute devices to change, based on the performance, a configuration of the first machine learning model and a configuration of the second machine learning model between the in-line mode and the out-of-band mode; and
deploying, to the one or more compute devices via the network, a third machine learning model different from the first machine learning model and the second machine learning model from the plurality of trained machine learning models to operate in the in-line mode, causing the one or more compute devices to initiate the remedial action when a statistically-combined classification value of a classification using the first machine learning model and a classification using the third machine learning model indicates a digital resource from the third plurality of digital resources is associated with a cyberattack from the plurality of cyberattacks.