Multiple instance learning models for cybersecurity using JavaScript object notation (JSON) training data
Techniques and architecture are described for converting tree structured data such as, for example, JavaScript Object Notation (JSON) data, into multiple feature vectors to train multiple instance learning (MIL) models for providing cybersecurity in networks. In particular, a data set is provided, wherein the data set comprises a sample configured as a hierarchal tree. The sample is converted into a set of path and value pairs, e.g., flattened into a set of path and value pairs, where the path is a sequence of field names and array indices encoding a position of a value. Each path and value pair of the set of path and value pairs is converted into a respective feature vector to form a set of feature vectors. The set of feature vectors is used to train a multiple instance learning (MIL) model, wherein each feature vector has a same, fixed length.
1 . A method comprising:
providing a data set, wherein the data set comprises a sample configured as a hierarchal tree;
based at least in part on the hierarchal tree, converting the sample into a set of path and value pairs;
converting each path and value pair of the set of path and value pairs into a respective feature vector to form a set of feature vectors, wherein each feature vector of the set of feature vectors has a fixed length identical to other feature vectors of the set of feature vectors, and wherein the fixed length corresponds to a predetermined total number of path and feature values of the path and value pairs;
training one or more multiple instance learning models (MIL) using the set of feature vectors to provide one or more trained MIL models;
analyzing, using the one or more trained MIL models, traffic within a network; and
based at least in part on the analyzing, determining whether to forward the traffic on to a network device.
2 . The method of claim 1 , wherein the data set comprises a JavaScript Object Notation (JSON) data set.
3 . The method of claim 1 , wherein the data set comprises an Extensible Markup Language (XML) data set.
4 . The method of claim 3 , further comprising:
prior to converting the sample into the set of path and value pairs, converting the XML data set into a JavaScript Object Notation (JSON) data set.
5 . The method of claim 1 , wherein the path of each path and value pair comprises a sequence of field names and array indices encoding a respective position of a respective value within a respective hierarchal tree.
6 . The method of claim 5 , wherein each respective value consists of one of a string, a number, a Boolean value type, or a null value type.
7 . The method of claim 1 , wherein converting each path and value pair into the respective feature vector comprises using a path feature extractor to one or more of (i) extract a length of the path, (ii) extract a number of field names in the path, (iii) extract a number of arrays in the path, (iv) extract an index in a first array or a last array, (v) one-hot encoding for a predefined list of field names, or (vi) hashing an arbitrary field name into a fixed range of integers.
8 . The method of claim 1 , wherein converting each path and value pair into the respective feature vector comprises using a path feature extractor to one or more of (i) determine if a value is null type, (ii) determine if a value is Boolean type, (iii) determine if a value is Boolean type and equals true or false, (iv) determine if a value is a number, (v) determine if the value is a number value type and a value itself, (vi) determine if a value is string type, (vii) determine a length of a string if a value is string type, (viii) determine one-hot encoding for a predefined list of strings, or (ix) hashing an arbitrary string into a fixed range of integers.
9 . A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform actions comprising:
providing a data set, wherein the data set comprises a sample configured as a hierarchal tree;
based at least in part on the hierarchal tree, converting the sample into a set of path and value pairs;
converting each path and value pair of the set of path and value pairs into a respective feature vector to form a set of feature vectors, wherein each feature vector of the set of feature vectors has a fixed length identical to other feature vectors of the set of feature vectors, and wherein the fixed length corresponds to a predetermined total number of path and feature values of the path and value pairs;
training one or more multiple instance learning models (MIL) using the set of feature vectors to provide one or more trained MIL models;
analyzing, using the one or more trained MIL models, traffic within a network; and
based at least in part on the analyzing, determining whether to forward the traffic on to a network device.
10 . The system of claim 9 , wherein the data set comprises a JavaScript Object Notation (JSON) data set.
11 . The system of claim 9 , wherein the data set comprises an Extensible Markup Language (XML) data set.
12 . The system of claim 11 , wherein the actions further comprise:
prior to converting the sample into the set of path and value pairs, converting the XML data set into a JavaScript Object Notation (JSON) data set.
13 . The system of claim 9 , wherein the path of each path and value pair comprises a sequence of field names and array indices encoding a respective position of a respective value within a respective hierarchal tree.
14 . The system of claim 13 , wherein each respective value consists of one of a string, a number, a Boolean value type, or a null value type.
15 . The system of claim 9 , wherein converting each path and value pair into the respective feature vector comprises using a path feature extractor to one or more of (i) extract a length of the path, (ii) extract a number of field names in the path, (iii) extract a number of arrays in the path, (iv) extract an index in a first array or a last array, (v) one-hot encoding for a predefined list of field names, or (vi) hashing an arbitrary field name into a fixed range of integers.
16 . The system of claim 9 , wherein converting each path and value pair into the respective feature vector comprises using a path feature extractor to one or more of (i) determine if a value is null type, (ii) determine if a value is Boolean type, (iii) determine if a value is Boolean type and equals true or false, (iv) determine if a value is a number, (v) determine if the value is a number value type and a value itself, (vi) determine if a value is string type, (vii) determine a length of a string if a value is string type, (viii) determine one-hot encoding for a predefined list of strings, or (ix) hashing an arbitrary string into a fixed range of integers.
17 . One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform actions comprising:
providing a data set, wherein the data set comprises a sample configured as a hierarchal tree;
based at least in part on the hierarchal tree, converting the sample into a set of path and value pairs;
converting each path and value pair of the set of path and value pairs into a respective feature vector to form a set of feature vectors, wherein each feature vector of the set of feature vectors has a fixed length identical to other feature vectors of the set of feature vectors, and wherein the fixed length corresponds to a predetermined total number of path and feature values of the path and value pairs;
training one or more multiple instance learning models (MIL) using the set of feature vectors to provide one or more trained MIL models;
analyzing, using the one or more trained MIL models, traffic within a network; and
based at least in part on the analyzing, determining whether to forward the traffic on to a network device.
18 . The one or more non-transitory computer-readable media of claim 17 , wherein the data set comprises a JavaScript Object Notation (JSON) data set.