Reduction of computation complexity of neural network sensitivity analysis
As part of neural network sensitivity analyses, base outputs of hidden layer nodes of a neural network model for non-perturbed variables can be reused when perturbing the variables. Such an arrangement greatly reduces complexity of the calculations required to generate outputs of the model. Related apparatus, systems, techniques and articles are also described.
1. A method for implementation by one or more data processors of at least one computing system comprising:
inputting an input variable for an input layer node forming part of a neural network, the input variable being extracted from a record, the input variable being input into the input layer node to generate a base output for a first hidden layer node forming part of the neural network and to generate a non-perturbed output of the neural network;
caching the generated base output for the first hidden layer node;
perturbing, for the record, the input variable to generate a perturbed input layer output of the input layer node;
generating, in response to the caching and for the record, a perturbed hidden layer output for the first hidden layer node by reusing the generated base output and the perturbed input layer output;
initiating sensitivity analyses of the neural network sensitivity by comparing the non-perturbed output with the perturbed hidden layer output.
2. A method as in claim 1 , wherein the base outputs a i are determined using:
a
i
=
∑
j
=
1
N
w
ij
x
j
+
b
i
where i is an index of the hidden layer nodes, j is an index of input variables, x is the input variable, w ij is weight from input node j to hidden node i, and b i is bias for hidden node i.
3. A method as in claim 2 , wherein for a perturbed input variable k that is perturbed by A, the output for the hidden layer node that reuses the corresponding cached generated base output uses:
a
i
(
k
)
=
∑
j
=
1
N
w
ij
x
j
+
b
i
+
Δ
x
k
*
w
ik
=
a
i
+
Δ
x
k
*
w
ik
4. A method as in claim 1 , wherein at least one of the inputting, perturbing, generating, and initiating is implemented by at least one data processor.
5. A non-transitory computer program product storing instructions which when executed by at least one data processor of at least one computing system result in operations comprising:
inputting an input variable for an input layer node forming part of a neural network, the input variable being extracted from a record, the input variable being input into the input layer node to generate a base output for a first hidden layer node forming part of the neural network and to generate a non-perturbed output of the neural network;
caching the generated base output for the first hidden layer node;
perturbing, for the record, the input variable to generate a perturbed input layer output of the input layer node;
generating, in response to the caching and for the record, a perturbed hidden layer output for the first hidden layer node by reusing the generated base output and the perturbed input layer output;
initiating sensitivity analyses of the neural network sensitivity by comparing the non-perturbed output with the perturbed hidden layer output.
6. A computer program product as in claim 5 , wherein the base outputs a i are determined using:
a
i
=
∑
j
=
1
N
w
ij
x
j
+
b
i
where i is an index of the hidden layer nodes, j is an index of input variables, x is the input variable, w ij is weight from input node j to hidden node i, and b i is bias for hidden node i.
7. A computer program product as in claim 6 , wherein for a perturbed input variable k that is perturbed by Δ, the output for the hidden layer node that reuses the corresponding cached generated base output uses:
a
i
(
k
)
=
∑
j
=
1
N
w
ij
x
j
+
b
i
+
Δ
x
k
*
w
ik
=
a
i
+
Δ
x
k
*
w
ik
8. A computer program product as in claim 5 , further comprising:
caching the generated base output for the hidden layer node;
wherein the cached generated base output are reused.
9. A computer program product as in claim 5 , wherein at least one of the inputting, perturbing, generating, and initiating is implemented by at least one data processor.
10. A computer program product as in claim 5 , further comprising:
generating an array of perturbed scores, the perturbed scores based on a single input variable perturbation that resulted in the generated perturbed hidden layer output; and
comparing the array of perturbed scores to non-perturbed scores based on the non-perturbed output to determine the sensitivity of the neural network.
11. A method for analyzing the sensitivity of a neural network implemented by one or more data processors of at least one computing system comprising:
generating, by at least one of the one or more data processors, a base output for a first hidden node of a neural network using a non-perturbed input variable;
caching the generated base output for the first hidden layer node;
generating, in response to the caching, an output of the neural network using a perturbed input variable and by reusing the cached base output; and
providing data comprising the output.
12. The method of claim 11 , wherein providing the data comprises at least one of: displaying the data, transmitting the data to a remote computing device, loading the data into memory, and storing the data.