Adaptive detection of security threats through training of computer-implemented models
A generated training set comprising a plurality of training samples is received. The generated training set includes at least one training sample constructed using one or more linguistic hints, comprising at least one keyword of phrase, about an attack for which malicious textual communications associated with the attack, when processed by a natural language processing model could be classified as benign textual communications before being trained using the generated training set. The natural language processing model is trained at least in part by using the generated training set, wherein the trained natural language processing model is configured to determine a likelihood that a received communication transmitted by a sender to a recipient poses a risk.
1 . A system, comprising:
a processor configured to:
receive a user report identifying at least one malicious communication that was previously misclassified as a benign communication by a natural language processing model;
based at least in part in response to the user report associated with the misclassification by the natural language processing model, generate a training set comprising a plurality of training samples, wherein the generated training set includes at least one training sample constructed using one or more linguistic hints derived from the user-reported at least one malicious communication that was previously misclassified as benign, comprising at least one keyword or phrase, about an attack for which malicious textual communications associated with the attack, when processed by the natural language processing model, could be classified as benign textual communications before being trained using the generated training set; and
further train the natural language processing model at least in part by using the generated training set, wherein the trained natural language processing model is configured to determine a likelihood that a received communication transmitted by a sender to a recipient poses a risk; and
a memory coupled to the processor and configured to provide the processor with instructions.
2 . The system of claim 1 , wherein the natural language processing model is trained in response to a report of a malicious email.
3 . The system of claim 1 , wherein the natural language processing model is trained in response to a specification of a novel attack.
4 . The system of claim 1 , wherein the natural language processing model is trained in response to an indication that a threshold quantity of false negatives is available for training purposes.
5 . The system of claim 1 , wherein the generated training set is generated including by performing a text augmentation on an exemplar malicious message.
6 . The system of claim 5 , wherein performing the text augmentation includes programmatically altering individual characters of the exemplar malicious message.
7 . The system of claim 5 , wherein performing the text augmentation includes programmatically altering one or more words of the exemplar malicious message.
8 . The system of claim 5 , wherein performing the text augmentation includes programmatically altering one or more clauses of the exemplar malicious message.
9 . The system of claim 5 , wherein performing the text augmentation includes replacing an element of the exemplar malicious message with a semantic equivalent.
10 . The system of claim 9 , wherein the semantic equivalent comprises a word synonym.
11 . The system of claim 9 , wherein the semantic equivalent comprises a phrase.
12 . The system of claim 1 , wherein the generated training set of training samples is generated including by modifying a predetermined amount of an exemplar malicious message.
13 . The system of claim 1 , wherein training the natural language processing model includes training a word-level convolutional neural network that predicts an attack label of an email.
14 . The system of claim 1 , wherein training the natural language processing model includes training a character-level convolutional neural network that predicts an attack label of an email.
15 . The system of claim 1 , wherein the processor is further configured to take a remedial action in response to determining that the received communication poses the risk.
16 . A method, comprising:
receiving a user report identifying at least one malicious communication that was previously misclassified as a benign communication by a natural language processing model;
based at least in part in response to the user report associated with the misclassification by the natural language processing model, generating a training set comprising a plurality of training samples, wherein the generated training set includes at least one training sample constructed using one or more linguistic hints derived from the user-reported at least one malicious communication that was previously misclassified as benign, comprising at least one keyword or phrase, about an attack for which malicious textual communications associated with the attack, when processed by the natural language processing model, could be classified as benign textual communications before being trained using the generated training set; and
further training the natural language processing model at least in part by using the generated training set, wherein the trained natural language processing model is configured to determine a likelihood that a received communication transmitted by a sender to a recipient poses a risk.
17 . The method of claim 16 , wherein training the natural language processing model includes training a word-level convolutional neural network that predicts an attack label of an email.
18 . The method of claim 16 , wherein training the natural language processing model includes training a character-level convolutional neural network that predicts an attack label of an email.
19 . The method of claim 16 , further comprising taking a remedial action in response to determining that the received communication poses the risk.
20 . A computer program product, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:
receiving a user report identifying at least one malicious communication that was previously misclassified as a benign communication by a natural language processing model;
based at least in part in response to the user report associated with the misclassification by the natural language processing model, generating a training set comprising a plurality of training samples, wherein the generated training set includes at least one training sample constructed using one or more linguistic hints derived from the user-reported at least one malicious communication that was previously misclassified as benign, comprising at least one keyword or phrase, about an attack for which malicious textual communications associated with the attack, when processed by the natural language processing model, could be classified as benign textual communications before being trained using the generated training set; and
further training the natural language processing model at least in part by using the generated training set, wherein the trained natural language processing model is configured to determine a likelihood that a received communication transmitted by a sender to a recipient poses a risk.