Reduction of computation complexity of neural network sensitivity analysis
As part of neural network sensitivity analysis, base outputs of hidden layer nodes of a neural network model for non-perturbed variables can be reused when perturbing the variables. Such an arrangement greatly reduces complexity of the calculations required to generate outputs of the model. Related apparatus, systems, techniques and articles are also described.
1. A method for analyzing the sensitivity of a neural network implemented by one or more data processors of at least one computing system comprising:
receiving, by at least one of the one or more data processors, a plurality of records, each record comprising input variables for a plurality of input layer nodes forming part of a neural network;
inputting, for each record, the input variables into the input layer nodes to generate a base output for each of a plurality of first hidden layer nodes forming part of the neural network and to generate a non-perturbed output of the neural network;
caching the generated base outputs for each of the plurality of the first hidden layer nodes;
perturbing, for each record, each input variable;
generating, for each record, an output for the first hidden layer nodes by reusing the corresponding cached base outputs and a perturbed output of the neural network; and
initiating sensitivity analysis of the neural network sensitivity by comparing the non-perturbed outputs with the perturbed outputs across the input layer nodes.
2. A method as in claim 1 , wherein the base outputs a i are determined using:
a
i
=
∑
j
=
1
N
w
ij
x
j
+
b
i
where i is an index of the first hidden layer nodes, j is an index of input variables, x is the input variable, w ij is weight from input node j to hidden node i, and b i is bias for hidden node i.
3. A method as in claim 2 , wherein for a perturbed input variable k that is perturbed by Δ, the output for the first hidden layer nodes that reuses the corresponding cached generated base output uses:
a
i
(
k
)
=
∑
j
=
1
N
w
ij
x
j
+
b
i
+
Δ
x
k
*
w
ik
=
a
i
+
Δ
x
k
*
w
ik
4. The method of claim 1 , further comprising:
generating an array of perturbed scores, the perturbed scores based on a single input variable perturbation that resulted in the generated output; and
comparing the array of perturbed scores to scores based on the non-perturbed outputs to determine the sensitivity of the neural network.
5. A non-transitory computer program product storing instructions which when executed by at least one data processor of at least one computing system result in operations comprising:
receiving a plurality of records, each record comprising input variables for a plurality of input layer nodes forming part of a neural network;
inputting, for each record, the input variables into the input layer nodes to generate a base output for each of a plurality of first hidden layer nodes forming part of the neural network and to generate a non-perturbed output of the neural network;
caching the generated base outputs for each of the plurality of the first hidden layer nodes;
perturbing, for each record, each input variable;
generating, for each record, an output for the first hidden layer nodes by reusing the corresponding cached base outputs and a perturbed output of the neural network; and
initiating sensitivity analysis of the neural network sensitivity by comparing the non-perturbed outputs with the perturbed outputs across the input layer nodes.
6. A non-transitory computer program product as in claim 5 , wherein the base outputs a i are determined using:
a
i
=
∑
j
=
1
N
w
ij
x
j
+
b
i
where i is an index of the first hidden layer nodes, j is an index of input variables, x is the input variable, w ij is weight from input node j to hidden node i, and b i is bias for hidden node i.
7. A non-transitory computer program product as in claim 6 , wherein for a perturbed input variable k that is perturbed by Δ, the output for the first hidden layer nodes that reuses the corresponding cached generated base output uses:
a
i
(
k
)
=
∑
j
=
1
N
w
ij
x
j
+
b
i
+
Δ
x
k
*
w
ik
=
a
i
+
Δ
x
k
*
w
ik
8. A method for analyzing the sensitivity of a neural network implemented by one or more data processors of at least one computing system comprising:
generating, by at least one of the one or more data processors, base outputs for each of a plurality of first hidden nodes of a neural network using non-perturbed input variables;
caching the generated base outputs for each of the plurality of the first hidden layer nodes;
generating an output of the neural network using perturbed input variables and by reusing the cached base outputs; and
providing data comprising the output.
9. The method of claim 8 , wherein providing data comprises at least one of: displaying the data, transmitting the data to a remote computing device, loading the data into memory, and storing the data.