Apparatuses and methods for text classification
Apparatuses and methods are provided for classifying textual content using a text classifier for determining to which class the textual content belongs. After classification, the text classifier provides the classification result and a context relevant to the classification result to an explanation system. The explanation system predicts, from the classification result and the context relevant to the classification result, one or more reasons behind the classification result.
1. A method, comprising:
receiving a text input to be classified;
predicting, using a text classifier, a class of the text input, to obtain a prediction result;
extracting a context relevant to the prediction result by identifying the context relevant to the prediction result by selecting words of the text input that are relevant to the prediction result using an interpretive multi-head attention module configured to perform;
identifying a multi-head attention importance of individual words of the text input as a normalized aggregate over attention head weights from words to the prediction result,
determining an adversarial importance of the individual words, by each respective word of the text input, by removing the respective word in the text input and computing a prediction probability of the text input when the respective word is removed,
applying a gradient-based filter defined as a normalized gradient for the individual words with respect to the prediction result, and
determining a weighted average of the multi-head attention importance, the adversarial importance, and the normalized gradient to generate the extracted context;
determining one or more reasons for the prediction result based on the extracted context, wherein the one or more reasons explain why the text input was classified into the predicted class; and
providing the prediction result and the determined one or more reasons as a classification result.
2. The method according to claim 1 , wherein determining the one or more reasons for the prediction result comprises:
determining, using a machine learning arrangement, the one or more reasons for the prediction result based on the identified context, using a reason classifier and a knowledge base configured to be used to predict a reason for classification.
3. The method according to claim 2 , wherein predicting the reason for classification comprises expanding the identified context using a knowledge base comprising semantical relationships of words.
4. The method according to claim 1 , further comprising computing a value representing a confidence of the prediction result and the determined one or more reasons.
5. The method according to claim 4 , further comprising:
comparing the computed value against a threshold; and
forwarding the text input, the prediction result, and the one or more reasons to a system operator when the computed value is lower than the threshold.
6. The method according to claim 1 , further comprising generating an explanation based on the one or more reasons.
7. The method according to claim 1 , wherein the text classifier is a language-representation based neural network.
8. An apparatus, comprising:
processing circuitry configured to:
receive a text input to be classified;
predict, using a text classifier, a class of the text input, to obtain a prediction result;
extract a context relevant to the prediction result by identifying the context relevant to the prediction result by selecting words of the text input that are relevant to the prediction result using an interpretive multi-head attention module, wherein the interpretive multi-head attention module is configured to:
identify a multi-head attention importance of individual words of the text input as a normalized aggregate over attention head weights from words to the prediction result,
determine an adversarial importance of the individual words, by each respective word of the text input, by removing the respective word in the text input and computing a prediction probability of the text input when the respective word is removed,
apply a gradient-based filter defined as a normalized gradient for the individual words with respect to the prediction result, and
determine a weighted average of the multi-head attention importance, the adversarial importance, and the normalized gradient to generate the extracted context;
determine one or more reasons for the prediction result based on the extracted context, wherein the one or more reasons explain why the text input was classified into the predicted class; and
provide the prediction result and the determined one or more reasons as a classification result.
9. The apparatus according to claim 8 , wherein the processing circuitry is configured to determine the one or more reasons for the prediction result by determining, using a machine learning arrangement, the one or more reasons for the prediction result based on the identified context and using a reason classifier and a knowledge base configured to be used to predict reasons for classification.
10. The apparatus according to claim 8 , wherein the processing circuitry is further configured to:
expand the identified context using a knowledge base comprising semantical relationships of words.
11. The apparatus according to claim 8 , wherein the processing circuitry is further configured to:
compute a value representing a confidence of the prediction result and the determined one or more reasons.
12. The apparatus according to claim 11 , wherein the processing circuitry is further configured to:
compare the computed value against a threshold; and
forward the text input, the prediction result, and the one or more reasons to a system operator when the computed value is lower than the threshold.
13. The apparatus according to claim 8 , wherein the processing circuitry is further configured to generate an explanation based on the one or more reasons.
14. An apparatus, comprising:
at least one processor; and
a non-transitory computer readable storage medium storing a program that is executable by the at least one processor, the program including instructions to:
receive a text input to be classified;
predict, using a text classifier, a class of the text input, to obtain a prediction result;
extract a context relevant to the prediction result by identifying the context relevant to the prediction result by selecting words of the text input that are relevant to the prediction result using an interpretive multi-head attention module, wherein the interpretive multi-head attention module is configured to:
identify a multi-head attention importance of individual words of the text input as a normalized aggregate over attention head weights from words to the prediction result,
determine an adversarial importance of the individual words, by each respective word of the text input, by removing the respective word in the text input and computing a prediction probability of the text input when the respective word is removed,
apply a gradient-based filter defined as a normalized gradient for the individual words with respect to the prediction result, and
determine a weighted average of the multi-head attention importance, the adversarial importance, and the normalized gradient to generate the extracted context;
determine one or more reasons for the prediction result based on the extracted context, wherein the one or more reasons explain why the text input was classified into the predicted class; and
provide the prediction result and the determined one or more reasons as a classification result.
15. The apparatus according to claim 14 , wherein the program includes instructions to determine the one or more reasons for the prediction result by determining, using a machine learning arrangement, the one or more reasons for the prediction result based on the identified context and using a reason classifier and a knowledge base configured to be used to predict reasons for classification.
16. The apparatus according to claim 14 , wherein the program further includes instructions to:
expand the identified context using a knowledge base comprising semantical relationships of words.
17. The apparatus according to claim 14 , wherein the program further includes instructions to:
compute a value representing a confidence of the prediction result and the determined one or more reasons.