Compact representation and time series segment retrieval through deep learning
Systems and methods for retrieving similar multivariate time series segments are provided. The systems and methods include extracting a long feature vector and a short feature vector from a time series segment, converting the long feature vector into a long binary code, and converting the short feature vector into a short binary code. The systems and methods further include obtaining a subset of long binary codes from a binary dictionary storing dictionary long codes based on the short binary codes, and calculating similarity measure for each pair of the long feature vector with each dictionary long code. The systems and methods further include identifying a predetermined number of dictionary long codes having the similarity measures indicting a closest relationship between the long binary codes and dictionary long codes, and retrieving a predetermined number of time series segments associated with the predetermined number of dictionary long codes.
1 . A computer-implemented method of retrieving similar multivariate time series segments, comprising:
converting long feature vectors and short feature vectors extracted from multivariate time series segments into long binary codes and short binary codes, respectively;
training a neural network that preserves similarity measure relations from the short binary codes and the long binary codes with triplet losses and cross-entropy loss to utilize label information for capturing a global decision boundary in a feature space from the multivariate time series segments including:
computing a first Hamming distance between an output value of a hash function for an anchor class label and an output value of the hash function for a positive class label;
computing a second Hamming distance between the output value of the hash function for the anchor class label and an output value of the hash function for a negative class label;
computing the triplet losses based on a maximum between zero and a sum of a margin value and a difference of the first Hamming distance and the second Hamming distance;
computing the cross-entropy loss based on a product of a one-hot representation of a class label and a logarithm of a predicted probability for a segment from the multivariate time series segments, and a one vector with a class label set length;
obtaining, with the neural network, a subset of long binary codes from a binary dictionary storing dictionary long codes based on the short binary codes;
calculating similarity measures for each pair of the long feature vectors with each dictionary long codes;
identifying, with the neural network, a predetermined number of dictionary long codes having the similarity measures indicating a closest relationship between the long binary codes and the dictionary long codes; and
reducing memory complexity and usage by compressing corresponding data for the multivariate time series segments for downstream tasks by retrieving a predetermined number of time series segments associated with the predetermined number of dictionary long codes.
2 . The computer-implemented method as recited in claim 1 , wherein training the neural network further comprises minimizing a sum of the triplet losses relaxed to be continuous and differentiable, and the cross-entropy loss.
3 . The computer-implemented method as recited in claim 1 , wherein the long feature vectors and the short feature vectors are extracted from the multivariate time series segments using a long short term memory (LSTM).
4 . The computer-implemented method as recited in claim 3 , wherein the long feature vectors are converted into the long binary codes by checking signs of all entries in the long feature vectors.
5 . The computer-implemented method as recited in claim 4 , wherein the short feature vectors are converted into the short binary codes by classifying the short binary codes to classes with a linear mapping.
6 . The computer-implemented method as recited in claim 2 , wherein training the neural network further comprises compressing intermediate features from the class labels to desired dimensional features with a classification layer.
7 . The computer-implemented method as recited in claim 5 , wherein classifying involves computing, with a softmax layer, a probability of the short binary codes belonging to each of a plurality of labels associated with the multivariate time series segments.
8 . A processing system for retrieving similar multivariate time series segments, comprising:
one or more processors;
a memory coupled to the one or more processors;
a long feature extractor stored in the memory, wherein the long feature extractor is configured to extract long feature vectors from multivariate time series segments;
a short feature extractor stored in the memory, wherein the short feature extractor is configured to convert the long feature vectors generated by the long feature extractor into shorter length features through a linear mapping;
a long binary extractor stored in the memory, wherein the long binary extractor is configured to convert the long feature vectors from the long feature extractor into long binary codes having a same length as the long feature vectors;
a short binary extractor stored in the memory, wherein the short binary extractor is configured to convert short feature vectors from the short feature extractor into short binary codes having the same length as the short feature vectors;
a neural network trained to preserve similarity measure relations from the short binary codes and the long binary codes with triplet losses based on a maximum between zero and a sum of a margin value and a difference of a first Hamming distance between an output value of a hash function for an anchor class label and an output value of the hash function for a positive class label and a second Hamming distance between the output value of the hash function for the anchor class label and an output value of the hash function for a negative class label and cross-entropy loss based on a product of a one-hot representation of a class label and a logarithm of a predicted probability for a segment from the multivariate time series segments, and a one vector with a class label set length, to utilize label information for capturing a global decision boundary in a feature space from the multivariate time series segments; and
a similarity comparator stored in the memory, wherein the similarity comparator is configured to calculate a pairwise similarity measures between the long binary codes extracted from query and the long binary codes retrieved from a dictionary, and reduces memory complexity and usage by compressing corresponding data for the multivariate time series segments for downstream tasks by identifying a predetermined number of dictionary long codes having the pairwise similarity measures indicating a closest relationship between the long binary codes and dictionary long codes.
9 . The processing system as recited in claim 8 , wherein the short feature vectors from the short feature extractor are converted into the short binary codes having the same length as the short feature vectors by checking signs of entries in the short feature vectors.
10 . The processing system as recited in claim 8 , wherein the similarity comparator is configured to retrieve a predetermined number of time series segments associated with the predetermined number of dictionary long codes, and display the predetermined number of time series segments to a user.
11 . The processing system as recited as recited in claim 10 , wherein the long feature vectors and the short feature vectors are extracted from the multivariate time series segments using a long short term memory (LSTM).
12 . The processing system as recited as recited in claim 11 , wherein the long feature vectors are converted into the long binary codes by checking signs of all entries in the long feature vectors.
13 . The processing system as recited as recited in claim 12 , wherein the short feature vectors are converted into the short binary codes by a linear mapping.
14 . The processing system as recited as recited in claim 13 , wherein the short binary extractor is further configured to classifying the short binary codes to classes.
15 . A computer program product for retrieving similar multivariate time series segments, the computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform a method comprising:
converting long feature vectors and short feature vectors extracted from multivariate time series segments into long binary codes and short binary codes, respectively;
training a neural network that preserves similarity measure relations from the short binary codes and the long binary codes with triplet losses and cross-entropy loss to utilize label information for capturing a global decision boundary in a feature space from the multivariate time series segments including:
computing a first Hamming distance between an output value of a hash function for an anchor class label and an output value of the hash function for a positive class label;
computing a second Hamming distance between the output value of the hash function for the anchor class label and an output value of the hash function for a negative class label;
computing the triplet losses based on a maximum between zero and a sum of a margin value and a difference of the first Hamming distance and the second Hamming distance;
computing the cross-entropy loss based on a product of a one-hot representation of a class label and a logarithm of a predicted probability for a segment from the multivariate time series segments, and a one vector with a class label set length;
obtaining, with the neural network, a subset of long binary codes from a binary dictionary storing dictionary long codes based on the short binary codes;
calculating similarity measures for each pair of the long feature vectors with each dictionary long codes;
identifying, with the neural network, a predetermined number of dictionary long codes having the similarity measures indicating a closest relationship between the long binary codes and the dictionary long codes; and
reducing memory complexity and usage by compressing corresponding data for the multivariate time series segments for downstream tasks by retrieving a predetermined number of time series segments associated with the predetermined number of dictionary long codes.
16 . The computer program product as recited in claim 15 , wherein to train the neural network further comprises updating parameters of the neural network through backpropagation based on an optimization objective function that utilizes criterions for codes obtained at a top layer of the neural network.
17 . The computer program product as recited in claim 15 , wherein the long feature vectors and the short feature vectors are extracted from the multivariate time series segments using a long short term memory (LSTM).
18 . The computer program product as recited in claim 17 , wherein the long feature vectors are converted into the long binary codes by checking signs of all entries in the long feature vectors.
19 . The computer program product as recited in claim 18 , wherein the short feature vectors are converted into the short binary codes by a linear mapping.
20 . The computer program product as recited in claim 19 , further comprising classifying the short binary codes to a class, wherein classifying involves computing a probability of the short binary codes belonging to each of a plurality of labels associated with the multivariate time series segments.