Anomaly detection systems and methods
Some embodiments employ a novel procedure of training an artificial intelligence system (e.g., set of deep neural networks) for anomaly detection in applications such as natural language processing and computer security. Token sequences selected from a training corpus are distorted according to at least one of a plurality of pre-determined sequence transformations, before being fed to a sequence analyzer. In turn, the sequence analyzer is trained to correctly guess which transformation was used to generate the respective input token sequence.
1. A computer-implemented anomaly detection method comprising employing at least one hardware processor of a computer system to:
in response to selecting a training token sequence from a training corpus of token sequences and in response to selecting a transformation from a pre-determined plurality of sequence transformations, distort the training token sequence by applying the selected transformation to the training token sequence to produce a distorted training token sequence;
input the distorted training sequence to a sequence analyzer, the sequence analyzer comprising:
a first prediction head configured to determine a first prediction indicator according to the distorted training token sequence, the first prediction indicator indicative of a likelihood that the selected transformation was applied to produce the distorted training token sequence, and
a second prediction head configured to determine a second prediction indicator according to the distorted training token sequence, the second prediction indicator indicative of a likelihood that a selected token of the training token sequence was changed by the application of the selected transformation;
in response to the sequence analyzer determining the first and second prediction indicators, adjust at least one parameter of the first prediction head according to the second prediction indicator and adjust at least one parameter of the second prediction head according to the first prediction indicator; and
in response to adjusting parameters of the first and second prediction heads, employ the sequence analyzer to determine whether a target token sequence is anomalous.
2. The method of claim 1 , wherein applying the selected transformation comprises replacing the selected token of the training token sequence with a substitute token.
3. The method of claim 2 , further comprising:
executing a token generator having a set of adjustable parameters and configured to generate the substitute token according to the training token sequence; and
in response to the sequence analyzer determining the first and second prediction indicators, adjusting at least one parameter of the set of adjustable parameters according to the first or second prediction indicator.
4. The method of claim 2 , further comprising adjusting the at least one parameter of the first prediction head or the at least one parameter of the second prediction head further according to a likelihood of appearance of the substitute token within the training token sequence in a context of the selected token.
5. The method of claim 1 , wherein applying the selected transformation comprises an item selected from a group consisting of deleting the selected token of the training token sequence, inserting an additional token into the training token sequence, and permuting a selected subset of tokens of the training token sequence.
6. The method of claim 1 , wherein:
the training and target token sequences comprise text formulated in a natural language; and
the method further comprises, in response to determining that the target token sequence is anomalous, determining that an author of the target token sequence differs from an author of the training token sequence.
7. The method of claim 1 , wherein:
the training and target token sequences comprise text formulated in a natural language; and
the method further comprises, in response to determining that the target token sequence is anomalous, determining that a subject matter of the target token sequence differs from a subject matter of the training token sequence.
8. The method of claim 1 , wherein:
the training and target token sequences comprise text formulated in a natural language; and
the method further comprises, in response to determining that the target token sequence is anomalous, determining that the target token sequence is machine-generated.
9. The method of claim 1 , wherein:
the training and target token sequences consist of sequences of computing events; and
the method further comprises, in response to determining that the target token sequence is anomalous, determining that the target token sequence is indicative of a computer security threat.
10. The method of claim 1 , comprising determining whether the target token sequence is anomalous according to a value of the second prediction indicator, the value of the second prediction indicator determined by the second prediction head according to the target token sequence.
11. A computer system comprising at least one hardware processor configured to execute an input modifier and a sequence analyzer, wherein:
the input modifier is configured to, in response to selecting a training token sequence from a training corpus of token sequences and in response to selecting a transformation from a pre-determined plurality of sequence transformations, distort the training token sequence by applying the selected transformation to the training token sequence to produce a distorted training token sequence;
the sequence analyzer comprises:
a first prediction head configured to determine a first prediction indicator according to the distorted training token sequence, the first prediction indicator indicative of a likelihood that the selected transformation was applied to produce the distorted training token sequence, and
a second prediction head configured to determine a second prediction indicator according to the distorted training token sequence, the second prediction indicator indicative of a likelihood that a selected token of the training token sequence was changed by the application of the selected transformation; and
wherein the at least one hardware processor is further configured to:
input the distorted token sequence into the sequence analyzer,
in response to the sequence analyzer determining the first and second prediction indicators, adjust at least one parameter of the first prediction head according to the second prediction indicator and adjust at least one parameter of the second prediction head according to the first prediction indicator, and
in response to adjusting parameters of the first and second prediction heads, employ the sequence analyzer to determine whether a target token sequence is anomalous.
12. The computer system of claim 11 , wherein applying the selected transformation comprises replacing the selected token of the training token sequence with a substitute token.
13. The computer system of claim 12 , wherein:
the input modifier comprises a token generator having a set of adjustable parameters and configured to generate the substitute token according to the training token sequence; and
the at least one hardware processor is further configured to, in response to the sequence analyzer determining the first and second prediction indicators, adjust at least one parameter of the set of adjustable parameters according to the first or second prediction indicator.
14. The computer system of claim 12 , wherein the at least one hardware processor is further configured to adjust the at least one parameter of the first prediction head or the at least one parameter of the second prediction head further according to a likelihood of appearance of the substitute token within the training token sequence in a context of the selected token.
15. The computer system of claim 11 , wherein applying the selected transformation comprises an item selected from a group consisting of deleting the selected token of the training token sequence, inserting an additional token into the training token sequence, and permuting a selected subset of tokens of the training token sequence.
16. The computer system of claim 11 , wherein:
the training and target token sequences comprise text formulated in a natural language; and
the at least one hardware processor is further configured to, in response to determining that the target token sequence is anomalous, determine that an author of the target token sequence differs from an author of the training token sequence.
17. The computer system of claim 11 , wherein:
the training and target token sequences comprise text formulated in a natural language; and
the at least one hardware processor is further configured to, in response to determining that the target token sequence is anomalous, determine that a subject matter of the target token sequence differs from a subject matter of the training token sequence.
18. The computer system of claim 11 , wherein:
the training and target token sequences comprise text formulated in a natural language; and
the at least one hardware processor is further configured to, in response to determining that the target token sequence is anomalous, determine that the target token sequence is machine-generated.
19. The computer system of claim 11 , wherein:
the training and target token sequences consist of sequences of computing events; and;
the at least one hardware processor is further configured to, in response to determining that the target token sequence is anomalous, determine that the target token sequence is indicative of a computer security threat.
20. The computer system of claim 11 , wherein the at least one hardware processor is configured to determine whether the target token sequence is anomalous according to a value of the second prediction indicator, the value of the second prediction indicator determined by the second prediction head according to the target token sequence.
21. A non-transitory computer-readable medium storing instructions which, when executed by at least one hardware processor of a computer system, cause the computer system to form an input modifier and a sequence analyzer, wherein:
the input modifier is configured to, in response to selecting a training token sequence from a training corpus of token sequences and in response to selecting a transformation from a pre-determined plurality of sequence transformations, distort the training token sequence by applying the selected transformation to the training token sequence to produce a distorted training token sequence;
the sequence analyzer comprises:
a first prediction head configured to determine a first output according to the distorted training token sequence, the first output indicative of a likelihood that the selected transformation was applied to produce the distorted training token sequence, and
a second prediction head configured to determine a second output according to the distorted training sequence, the second output indicative of a likelihood that a selected token of the training token sequence was changed by the application of the selected transformation; and
wherein the instructions further cause the computer system to:
input the distorted training sequence into the sequence analyzer,
in response to the sequence analyzer determining the first and second outputs, adjust at least one parameter of the first prediction head according to the first output and adjust at least one parameter of the second prediction head according to the second output, and
in response to adjusting parameters of the first and second prediction heads, employ the sequence analyzer to determine whether a target token sequence is anomalous.