Systems and methods for self supervised multi-view representation learning for time series
A multi-view contrastive relational learning framework is provided. In the multi-view contrastive relational learning framework, contrastive learning is augmented with a multi-view learning signal. The auxiliary views guide an encoder of the underlying time series data's main view, by using an inter-sample similarity structure as a learning signal to learn representations which encode information from multiple views.
1 . A method for multi-view contrastive relational learning for time-series prediction, the method comprising:
receiving, via a data interface, a training dataset of data samples of a plurality of time-series sequences;
generating a first set of samples with each sample corresponding to a respective time-step by transforming a time-series sequence of the plurality of time-series sequences via a first transformation;
generating a first set of pairs of random augmentations based on the first set of samples, wherein each pair of the first set of pairs of random augmentations includes random augmentations from the same time-step;
generating, by a main encoder, a set of pairs of representations of the first set of pairs of random augmentations;
generating a second set of samples by transforming the time-series sequence via a second transformation different from the first transformation;
generating a second set of pairs of random augmentations based on the second set of samples, wherein each pair of the second set of pairs of random augmentations includes random augmentations from the same time-step;
generating, by one or more auxiliary encoders, one or more sets of pairs of auxiliary representations of at least the second set of pairs of random augmentations;
computing a contrastive loss component based on similarities between representations from the set of pairs of representations;
computing a relational loss component by:
computing a first value based on a comparison of a random augmentation from the first set of pairs of random augmentations associated with a first time-step, and a random augmentation from the first set of pairs of random augmentations associated with a second time-step,
computing a second value based on a comparison of a random augmentation from the second set of pairs of random augmentations associated with the first time-step, and a random augmentation from the second set of pairs of random augmentations associated with the second time-step, and
using the second value as a soft-label for the first value in a cross-entropy loss;
computing a combined loss objective by combining the contrastive loss component and the relational loss component;
updating the main encoder based on the combined loss objective via backpropagation;
generating, via the updated main encoder, encoded data based on a measured set of time-series data; and
generating a prediction based on the encoded data.
2 . The method of claim 1 , further comprising:
generating, at each of a one or more auxiliary encoders, a respective contrastive loss objective based on similarities between representations from the one or more set of pairs of auxiliary representations; and
updating each of the one or more auxiliary encoders based on the respective contrastive loss respectively.
3 . The method of claim 1 , wherein each of the one or more auxiliary encoders corresponds to a specific view.
4 . The method of claim 1 , wherein the first transformation is identity transformation.
5 . The method of claim 1 , wherein the second transformation includes at least one of moving average, n-th order differencing, or discrete Fourier transformation.
6 . The method of claim 1 , wherein the contrastive loss component is generated by a non-parametric softmax function.
7 . The method of claim 1 , wherein the relational loss component is further based on a cross-entropy between intra-symbol pairs from the set of pairs of representations and intra-symbol pairs from the one or more sets of pairs of auxiliary representations.
8 . A system for multi-view contrastive relational learning for time-series prediction, the system comprising:
a non-transitory memory; and
one or more processors coupled to the non-transitory memory and configured to read instructions from the non-transitory memory to cause the system to perform operations comprising:
receiving, via a data interface, a training dataset of data samples of a plurality of time-series sequences;
generating a first set of samples with each sample corresponding to a respective time-step by transforming a time-series sequence of the plurality of time-series sequences via a first transformation;
generating a first set of pairs of random augmentations based on the first set of samples, wherein each pair of the first set of pairs of random augmentations includes random augmentations from the same time-step;
generating, by a main encoder, a set of pairs of representations of the first set of pairs of random augmentations;
generating a second set of samples by transforming the time-series sequence via a second transformation different from the first transformation;
generating a second set of pairs of random augmentations based on the second set of samples, wherein each pair of the second set of pairs of random augmentations includes random augmentations from the same time-step;
generating, by one or more auxiliary encoders, one or more sets of pairs of auxiliary representations of at least the second set of pairs of random augmentations;
computing a contrastive loss component based on similarities between representations from the set of pairs of representations;
computing a relational loss component by:
computing a first value based on a comparison of a random augmentation from the first set of pairs of random augmentations associated with a first time-step, and a random augmentation from the first set of pairs of random augmentations associated with a second time-step different from the first time-step,
computing a second value based on a comparison of a random augmentation from the second set of pairs of random augmentations associated with the first time-step, and a random augmentation from the second set of pairs of random augmentations associated with the second time-step, and
using the second value as a soft-label for the first value in a cross-entropy loss;
computing a combined loss objective by combining the contrastive loss component and the relational loss component;
updating the main encoder based on the combined loss objective via backpropagation;
generating, via the updated main encoder, encoded data based on a measured set of time-series data; and
generating a prediction based on the encoded data.
9 . The system of claim 8 , wherein the one or more processor is further configured to cause the system to perform operations comprising:
generating, at each of a one or more auxiliary encoders, a respective contrastive loss objective based on similarities between representations from the one or more set of pairs of auxiliary representations; and
updating each of the one or more auxiliary encoders based on the respective contrastive loss respectively.
10 . The system of claim 8 , wherein each of the one or more auxiliary encoders corresponds to a specific view.
11 . The system of claim 8 , wherein the first transformation is identity transformation.
12 . The system of claim 8 , wherein second transformation includes at least one of moving average, n-th order differencing, or discrete Fourier transformation.
13 . The system of claim 8 , wherein the contrastive loss component is generated by a non-parametric softmax function.
14 . The system of claim 8 , wherein the relational loss component is further based on a cross-entropy between intra-symbol pairs from the set of pairs of representations and intra-symbol pairs from the one or more sets of pairs of auxiliary representations.
15 . A non-transitory, machine-readable medium having stored thereon machine-readable instructions executable to cause a system to perform operations comprising:
receiving, via a data interface, a training dataset of data samples of a plurality of time-series sequences;
generating a first set of samples with each sample corresponding to a respective time-step by transforming a time-series sequence of the plurality of time-series sequences via a first transformation;
generating a first set of pairs of random augmentations based on the first set of samples, wherein each pair of the first set of pairs of random augmentations includes random augmentations from the same time-step;
generating, by a main encoder, a set of pairs of representations of the first set of pairs of random augmentations;
generating a second set of samples by transforming the time-series sequence via a second transformation different from the first transformation;
generating a second set of pairs of random augmentations based on the second set of samples, wherein each pair of the second set of pairs of random augmentations includes random augmentations from the same time-step;
generating, by one or more auxiliary encoders, one or more sets of pairs of auxiliary representations of at least the second set of pairs of random augmentations;
computing a contrastive loss component based on similarities between representations from the set of pairs of representations;
computing a relational loss component by:
computing a first value based on a comparison of a random augmentation from the first set of pairs of random augmentations associated with a first time-step, and a random augmentation from the first set of pairs of random augmentations associated with a second time-step different from the first time-step,
computing a second value based on a comparison of a random augmentation from the second set of pairs of random augmentations associated with the first time-step, and a random augmentation from the second set of pairs of random augmentations associated with the second time-step, and
computing the relational loss component by using the second value as a soft-label for the first value in a cross-entropy loss;
computing a combined loss objective by combining the contrastive loss component and the relational loss component;
updating the main encoder based on the combined loss objective via backpropagation;
generating, via the updated main encoder, encoded data based on a measured set of time-series data; and
generating a prediction based on the encoded data.
16 . The non-transitory, machine-readable medium of claim 15 , further having stored thereon machine-readable instructions executable to cause the system to perform operations comprising:
generating, at each of a one or more auxiliary encoders, a respective contrastive loss objective based on similarities between representations from the one or more set of pairs of auxiliary representations; and
updating each of the one or more auxiliary encoders based on the respective contrastive loss respectively.
17 . The non-transitory, machine-readable medium of claim 15 , wherein each of the one or more auxiliary encoders corresponds to a specific view.
18 . The non-transitory, machine-readable medium of claim 15 , wherein the first transformation is identity transformation.
19 . The non-transitory, machine-readable medium of claim 15 , wherein second transformation includes at least one of moving average, n-th order differencing, or discrete Fourier transformation.
20 . The non-transitory, machine-readable medium of claim 15 , wherein the relational loss component is further based on a cross-entropy between intra-symbol pairs from the set of pairs of representations and intra-symbol pairs from the one or more sets of pairs of auxiliary representations.