Machine learning outputs with high confidence explanations
A malware classification system provides improved confidence in explanations of neural network classification outputs using methods such as weighting or masking when training the neural network to train the network on a sample resembling or including the explanation. The explanation in some examples comprises a subset of a hierarchical input vector that is responsible for the neural network's classification output. In another example the neural network has an inner portion configured to reduce the weight of elements of the output not significantly contributing to the explanation of the output, such as by reducing the weight of as many such outputs to zero as is practical in generating the desired output.
1 . A method of generating an explanation of output from a machine-learning system, comprising:
receiving an input data string having a hierarchical structure;
analyzing the input data string using a machine learning module to automatically generate an output corresponding to the received input data string;
generating an explanation of the output comprising a subset of the input data string that is responsible for the output; and
training the machine learning module using a loss function optimized with an inner optimization that generates a sparse output corresponding to the explanation of the output, wherein the explanation comprises a subset of the input data string selected according to hierarchical groupings within the input data string, and wherein the training uses the explanation as a training input to update the machine learning module based on the hierarchical structure.
2 . The method of generating an explanation of output from a machine-learning system of claim 1 , further comprising applying a weighting or masking function when training the machine learning module using the generated explanation of the output and the output, configured to improve the output generated when the generated explanation of output is provided as input.
3 . The method of generating an explanation of output from a machine-learning system of claim 2 , wherein the loss function used to train the machine learning module is optimized using an inner optimization for an estimated output and the weighting or masking function.
4 . The method of generating an explanation of output from a machine-learning system of claim 3 , wherein the loss function optimization is trained using second-order stochastic gradient descent.
5 . The method of generating an explanation of output from a machine-learning system of claim 1 , further comprising constructing the machine learning module using a hierarchy of the input data string.
6 . The method of generating an explanation of output from a machine-learning system of claim 1 , wherein the machine learning module is a neural network.
7 . The method of generating an explanation of output from a machine-learning system of claim 6 , wherein the neural network comprises a hierarchical multiple-instance-learning neural network.
8 . The method of generating an explanation of output from a machine-learning system of claim 7 , wherein the neural network comprises an outer optimization of a loss function, and an inner optimization for one or more parameters of the loss function.
9 . The method of generating an explanation of output from a machine-learning system of claim 6 , wherein generating an explanation of the output is performed using logic other than the neural network.
10 . A method of generating an explanation of output from a machine-learning system, comprising:
receiving an input data string having a hierarchical structure;
analyzing the input data string using a machine learning module to automatically generate an output corresponding to the received input data string; and
generating an explanation of the output comprising a subset of the input data string that is responsible for the output;
the machine learning module comprising a neural network having an inner portion and an outer portion, wherein the inner portion is configured to reduce or zero the weight of elements of the output while still producing a desired output and wherein the outer portion aggregates a sparse output vector to produce a classification output, wherein the inner portion generates instance-level outputs that correspond to hierarchical segments of the input data string and are reduced or zeroed based on the hierarchical groupings, and wherein the outer portion aggregates the instance-level outputs according to the hierarchical structure.
11 . The method of generating an explanation of output from a machine-learning system, of claim 10 , wherein reducing or zeroing the weight of elements of the output while still producing a desired output comprises reducing the weight to zero.
12 . The method of generating an explanation of output from a machine-learning system, of claim 11 , wherein reducing or zeroing the weight of elements of the output while still producing a desired output to zero comprises reducing the weights of as many elements of the output to zero as can be achieved while still generating the desired output.
13 . The method of generating an explanation of output from a machine-learning system, of claim 10 , further comprising regularizing one or more outputs of the inner layer to be either zero or the largest observed value in an inner layer output vector comprising the one or more outputs of the inner layer.
14 . The method of generating an explanation of output from a machine-learning system of claim 10 , further comprising constructing the machine learning module using a hierarchy of the input data string.
15 . The method of generating an explanation of output from a machine-learning system of claim 10 , wherein the machine learning module is a neural network.
16 . The method of generating an explanation of output from a machine-learning system of claim 15 , wherein the neural network comprises a hierarchical multiple-instance-learning neural network.
17 . The method of generating an explanation of output from a machine-learning system of claim 15 , wherein the inner portion outputs instances of the input and the outer portion outputs aggregated output of the inner portion.
18 . The method of generating an explanation of output from a machine-learning system of claim 10 , wherein the inner portion and the outer portion of the machine learning module are configured to improve the confidence of the machine learning module in the explanation.
19 . The method of generating an explanation of output from a machine-learning system of claim 10 , wherein the inner portion and outer portion of the machine learning module are configured to reduce the size of explanation of output.
20 . A machine learning system, comprising:
a processor and a memory; and
a machine-readable medium with instructions stored thereon, the instructions when executed on the processor operable to cause the processor to:
receive an input data string having a hierarchical structure;
analyze the input data string using a machine learning module to automatically generate an output corresponding to the received input data string;
generate an explanation of the output comprising a subset of the input data string that is responsible for the output; and
train the machine learning module using the generated explanation of the output and the output, wherein the explanation is used as a training input and wherein the machine learning module comprises a neural network with an inner portion producing a sparse output corresponding to an explanation of the output and an outer portion aggregating the sparse output to produce the classification, wherein the inner portion produces a sparse output corresponding to an explanation that comprises a subset of the input data string selected according to hierarchical groupings within the input data string, and wherein the explanation is provided as a training input to update the neural network using the hierarchical structure of the input data string.