Concept-conditioned and pretrained language models based on time series to free-form text description generation
A computer-implemented method for employing a time-series-to-text generation model to generate accurate description texts is provided. The method includes passing time series data through a time series encoder and a multilayer perceptron (MLP) classifier to obtain predicted concept labels, converting the predicted concept labels, by a serializer, to a text token sequence by concatenating an aspect term and an option term of every aspect, inputting the text token sequence into a pretrained language model including a bidirectional encoder and an autoregressive decoder, and using adapter layers to fine-tune the pretrained language model to generate description texts.
1 . A computer-implemented method for employing a time-series-to-text generation model to generate accurate description texts, comprising:
passing time series data through a time series encoder and a multilayer perceptron (MLP) classifier to obtain predicted concept labels;
converting the predicted concept labels, by a serializer, to a text token sequence by concatenating an aspect term and an option term of every aspect;
inputting the text token sequence into a pretrained language model (PLM) including a bidirectional encoder and an autoregressive decoder;
iteratively training the PLM updated with adapter lavers by updating parameters of the adapter layers by minimizing a sum of a cross-entropy loss of the predicted concept labels and a text token maximum likelihood estimation (MLE) loss of the text token sequence; and
using the adapter layers to fine-tune the PLM to generate description texts.
2 . The computer-implemented method of claim 1 , wherein a concept word parser is used to mine phrases related to the aspects from a training text of the times series data.
3 . The computer-implemented method of claim 1 , wherein concepts serve as a ground truth for training concept classifiers and are aggregated to form a domain concept dictionary.
4 . The computer-implemented method of claim 1 , wherein iteratively training the PLM further comprises computing the cross-entropy loss by aggregating the cross-entropy loss between the predicted concept label and a ground truth label for every aspect.
5 . The computer-implemented method of claim 1 , wherein the MLP classifier takes a last hidden vector of the time series encoder to obtain the predicted concept labels based on a domain concept dictionary.
6 . The computer-implemented method of claim 1 , wherein iteratively training the PLM further comprises computing the text token MLE loss by aggregating a cross-entropy loss between a probability distribution output of the PLM and a ground-truth token at every position.
7 . The computer-implemented method of claim 1 , wherein the PLM is fine-tuned with schedule sampling using a combination of teacher-forcing and free-running modes that determines an input at a next position as a true token at a position in a training text for the teacher forcing mode and as a token sampled from a vocabulary based on an output distribution for the free-running mode.
8 . A computer program product for employing a time-series-to-text generation model to generate accurate description texts, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:
passing time series data through a time series encoder and a multilayer perceptron (MLP) classifier to obtain predicted concept labels;
converting the predicted concept labels, by a serializer, to a text token sequence by concatenating an aspect term and an option term of every aspect;
inputting the text token sequence into a pretrained language model (PLM) including a bidirectional encoder and an autoregressive decoder;
iteratively training the PLM updated with adapter layers by updating parameters of the adapter layers by minimizing a sum of a cross-entropy loss of the predicted concept labels and a text token maximum likelihood estimation (MLE) loss of the text token sequence; and
using the adapter layers to fine-tune the PLM to generate description texts.
9 . The computer program product of claim 8 , wherein a concept word parser is used to mine phrases related to the aspects from a training text of the times series data.
10 . The computer program product of claim 8 , wherein concepts serve as a ground truth for training concept classifiers and are aggregated to form a domain concept dictionary.
11 . The computer program product of claim 8 , wherein iteratively training the PLM further comprises computing the cross-entropy loss by aggregating the cross-entropy loss between the predicted concept label and a ground truth label for every aspect.
12 . The computer program product of claim 8 , wherein the MLP classifier takes a last hidden vector of the time series encoder to obtain the predicted concept labels based on a domain concept dictionary.
13 . The computer program product of claim 8 , wherein the adapter layers are inserted into all transformer layers of the PLM.
14 . The computer program product of claim 8 , wherein the PLM is fine-tuned with schedule sampling using a combination of teacher-forcing and free-running modes.
15 . A computer processing system for employing a time-series-to-text generation model to generate accurate description texts, comprising:
a memory device for storing program code; and
a processor device, operatively coupled to the memory device, for running the program code to:
pass time series data through a time series encoder and a multilayer perceptron (MLP) classifier to obtain predicted concept labels;
convert the predicted concept labels, by a serializer, to a text token sequence by concatenating an aspect term and an option term of every aspect;
input the text token sequence into a pretrained language model (PLM) including a bidirectional encoder and an autoregressive decoder;
iteratively train the PLM updated with adapter layers by updating parameters of the adapter layers by minimizing a sum of a cross-entropy loss of the predicted concept labels and a text token maximum likelihood estimation (MLE) loss of the text token sequence; and
use the adapter layers to fine-tune the PLM to generate description texts.
16 . The computer processing system of claim 15 , wherein a concept word parser is used to mine phrases related to the aspects from a training text of the times series data.
17 . The computer processing system of claim 15 , wherein concepts serve as a ground truth for training concept classifiers and are aggregated to form a domain concept dictionary.
18 . The computer processing system of claim 15 , wherein iteratively training the PLM further comprises computing the cross-entropy loss by aggregating the cross-entropy loss between the predicted concept label and a ground truth label for every aspect.
19 . The computer processing system of claim 15 , wherein the MLP classifier takes a last hidden vector of the time series encoder to obtain the predicted concept labels based on a domain concept dictionary.
20 . The computer processing system of claim 15 , wherein the adapter layers are inserted into all transformer layers of the pretrained language model.