METHOD AND SYSTEM FOR ANALYZING A NEURAL NETWORK
The disclosed relates to an analyzing of a neural network, in particular an already trained neural network, for example a recurrent neural network. For this purpose, input data to the neural network are stepwise increased. After each step, the probability for a specific classification of the neural network is analyzed. Upon the probability for the specific classification raises a predetermined threshold value, the data element which has been added last to the input data is identified to be relevant for the respective decision.
1 . A method for analyzing a neural network, the method comprising the steps of:
successively providing a stepwise increasing set of input data to the neural network;
receiving an output of the neural network in response to the provided input data, said output comprising a prediction probability for a specific classification;
comparing the prediction probability for the specific classification with a predetermined threshold value; and
identifying a characteristic element in the provided input data, if the prediction probability for the specific classification exceeds the predetermined threshold value.
2 . The method of claim 1 , wherein the neural network comprises a recurrent neural network.
3 . The method of claim 1 , wherein the input data comprise training data which have been used for training the neural network.
4 . The method of claim 1 , wherein the set of input data comprises a number of position indicators for indicating a target entity in the input data.
5 . The method of claim 1 , wherein the stepwise providing of input data comprises adding a further element of input data to previously provided input data.
6 . The method of claim 5 , wherein the identifying the characteristic element in the provided input data comprises identifying a number of added elements of input data which have been added in previous steps of providing input data.
7 . The method of claim 1 , wherein the neural network comprises a neural network configured to perform a semantic analysis.
8 . The method of claim 1 , wherein the set of input data comprise semantic data, in particular words or phrases.
9 . The method of claim 1 , wherein the characteristic element in the input data comprises a representative pattern responsible for a decision making of the neural network.
10 . A non-transient computer readable medium containing program instructions for causing a computer to perform a method of:
successively providing a stepwise increasing set of input data to the neural network;
receiving an output of the neural network in response to the provided input data, said output comprising a prediction probability for a specific classification;
comparing the prediction probability for the specific classification with a predetermined threshold value; and
identifying a characteristic element in the provided input data if the prediction probability for the specific classification exceeds the predetermined threshold value.
11 . A system for analyzing a neural network, the system comprising:
a data generator for successively providing a stepwise increasing set of input data to the neural network; and
an analyzing unit for receiving an output of the neural network in response to the provided input data, said output comprising a prediction probability for a specific classification, comparing the prediction probability for the specific classification with a predetermined threshold value, and identifying a characteristic element in the provided input data, if the prediction probability for the specific classification exceeds the predetermined threshold value.
12 . The system of claim 11 , wherein the neural network comprises a recurrent neural network.
13 . The system of claim 11 , wherein the input data comprise training data which have been used for training the neural network.
14 . The system of claim 11 , wherein the set of input data comprises a number of position indicators for indicating a target entity in the input data.
15 . The system of claim 11 , wherein said data generator is configured to add a further element of input data to previously provided input data.
16 . The system of claim 15 , wherein said analyzing unit is configured to identify the characteristic element in the provided input data by identifying a number of added elements of input data which have been added in previous steps of providing input data.
17 . The system of claim 11 , wherein the neural network comprises a neural network configured to perform a semantic analysis.
18 . The system of claim 11 , wherein said data generator is configured to provide input data comprising semantic data, in particular words or phrases.
19 . The system of claim 11 , wherein the characteristic element in the input data comprises a representative pattern responsible for a decision making of the neural network.