Method for training a classifier and system for classifying blocks
Blocks of spatially structured information such as log files or images are processed in a training loop for an attention-based classifier, using an active learning approach. First, the classifier provides a predicted label and an attention map for each classified block. Blocks are selected from the classified blocks if the output of the classifier for the respective block meets a selection criterion. The selected blocks are then displayed to a user together with the predicted label and a visual representation of the attention map. Based on these changes, the classifier is retrained. The method allows for an automatic, intelligent selection of a small number of data points that need to be labeled by a domain expert. The domain expert does not need to collect the training data a priori, but systematically and iteratively gets asked for training examples that are then directly used by the machine learning algorithm for learning.
1 . A computer implemented method for training a classifier, comprising the following operations performed by one or more processors:
processing, by one or more of the processors, blocks, with each block containing spatially structured information in the form of text and/or an image,
classifying, by one or more of the processors executing a classifier that uses an attention mechanism, each block, with the output of the classifier containing a predicted label and an attention map for each classified block,
selecting, by one or more of the processors, blocks from the classified blocks, if the output of the classifier for the respective block meets a selection criterion,
outputting, by a user interface, each selected block, wherein each selected block is displayed together with the predicted label and a visual representation of the attention map that the classifier has outputted,
detecting, by one or more of the processors, user interactions with the user interface, thereby receiving, by one or more of the processors, for at least one selected block a user-selected label and a user-selected attention map based on the user interactions, and
training, by one or more of the processors, the classifier with the at least one user-selected label and the at least one user-selected attention map,
wherein
the classifier is trained to perform automated log file diagnostics based on log entries received from components of a technical system, with the technical system being in particular a complex industrial system,
each block includes a sequence of log entries, with each log entry containing at least one timestamp and at least one message, and with the content of each block being processed as text tokens, and
for each selected block, the visual representation of the attention map is highlighting some of the text tokens of the selected block,
the classifier contains one or more convolutional neural networks with an attention mechanism,
the attention mechanism is a self-attention generative adversarial networks self-attention module,
each predicted label is a severity level of an event occurring in the technical system,
each attention map is a probability distribution over the text tokens contained in the respective block, and
for each selected block, each text token is highlighted in the visual representation if its probability value in the attention map exceeds a given threshold.
2 . The method according to claim 1 ,
wherein the steps of classifying, selecting, outputting, detecting, receiving and training are performed iteratively in a training loop.
3 . The method according to claim 1 ,
wherein the selection criterion is least confidence, margin sampling, and/or entropy sampling.
4 . Non-transitory computer-readable storage media having stored thereon
instructions executable by one or more processors of a computer system, wherein execution of the instructions causes the computer system to perform a method for training a classifier, the method comprising:
processing, by one or more of the processors, blocks, with each block containing spatially structured information in the form of text and/or an image;
classifying, by one or more of the processors executing a classifier that uses an attention mechanism, each block, with the output of the classifier containing a predicted label and an attention map for each classified block;
selecting, by one or more of the processors, blocks from the classified blocks, if the output of the classifier for the respective block meets a selection criterion;
outputting, by a user interface, each selected block, wherein each selected block is displayed together with the predicted label and a visual representation of the attention map that the classifier has outputted;
detecting, by one or more of the processors, user interactions with the user interface, thereby receiving, by one or more of the processors, for at least one selected block a user-selected label and a user-selected attention map based on the user interactions; and
training, by one or more of the processors, the classifier with the at least one user-selected label and the at least one user-selected attention map;
wherein
the classifier is trained to perform automated log file diagnostics based on log entries received from components of a technical system, with the technical system being in particular a complex industrial system,
each block includes a sequence of log entries, with each log entry containing at least one timestamp and at least one message, and with the content of each block being processed as text tokens, and
for each selected block, the visual representation of the attention map is highlighting some of the text tokens of the selected block,
the classifier contains one or more convolutional neural networks with an attention mechanism,
the attention mechanism is a self-attention generative adversarial networks self-attention module,
each predicted label is a severity level of an event occurring in the technical system,
each attention map is a probability distribution over the text tokens contained in the respective block, and
for each selected block, each text token is highlighted in the visual representation if its probability value in the attention map exceeds a given threshold.
5 . A computer program product, comprising a non-transitory computer readable hardware storage device having computer readable program code stored therein, said program code executable by one or more processors of a computer system to implement a method which is being executed by one or more processors of a computer system and performs the method for training a classifier, the method comprising:
processing, by one or more of the processors, blocks, with each block containing spatially structured information in the form of text and/or an image;
classifying, by one or more of the processors executing a classifier that uses an attention mechanism, each block, with the output of the classifier containing a predicted label and an attention map for each classified block;
selecting, by one or more of the processors, blocks from the classified blocks, if the output of the classifier for the respective block meets a selection criterion;
outputting, by a user interface, each selected block, wherein each selected block is displayed together with the predicted label and a visual representation of the attention map that the classifier has outputted;
detecting, by one or more of the processors, user interactions with the user interface, thereby receiving, by one or more of the processors, for at least one selected block a user-selected label and a user-selected attention map based on the user interactions; and
training, by one or more of the processors, the classifier with the at least one user-selected label and the at least one user-selected attention map;
wherein
the classifier is trained to perform automated log file diagnostics based on log entries received from components of a technical system, with the technical system being in particular a complex industrial system
each block includes a sequence of log entries, with each log entry containing at least one timestamp and at least one message, and with the content of each block being processed as text tokens, and
for each selected block, the visual representation of the attention map is highlighting some of the text tokens of the selected block,
the classifier contains one or more convolutional neural networks with an attention mechanism,
the attention mechanism is a self-attention generative adversarial networks self-attention module,
each predicted label is a severity level of an event occurring in the technical system,
each attention map is a probability distribution over the text tokens contained in the respective block, and
for each selected block, each text token is highlighted in the visual representation if its probability value in the attention map exceeds a given threshold.