Systems and methods of applying matrix product states to machine learning
A method of applying non-linear regression on a set of data points to get an estimate is described herein. The method includes receiving a set of N datapoints, separating the set of N datapoints into N b batches, receiving a family of fitting functions, and minimizing a log-cosh cost function for each batch by selecting parameters that minimize the log-cosh cost function. The parameters are obtained by: receiving a matrix product state (MPS) model and training the MPS to minimize loss over all the N b batches, including choosing an MPS with M+D tensors. All tensors except D correspond to one datapoint in each of the N b batches, D extra tensors in the MPS have a physical dimension of size M corresponding to the number of possible outputs for a given batch, and the coefficients of the tensors in the MPS minimize the log-cosh cost function sequentially over all the N b batches.
1 . A system for applying non-linear regression on a set of N datapoints to get an estimate, the system comprising at least one processor configured to:
receive the set of N datapoints, each of the datapoints having associated input values;
separate the set of N datapoints into N b batches, each of the N b batches having M=N/N b datapoints;
receive a family of fitting functions, each function comprising a dot product of a feature map Φ with a weight vector W, where the feature map Φ is local and represented by two-dimensional feature vectors; and
minimize a cost function for each of the N b batches by training a matrix product state (MPS) model to minimize loss over all the N b batches, the MPS model providing an MPS representation of the weight vector W and comprising M+D tensors,
wherein D corresponds to a dimension of a target function, and
wherein each of the tensors have respective coefficients, such that:
all tensors except D correspond to one datapoint i in each of the N b batches;
D extra tensors in the MPS have a physical dimension of size M corresponding to a number of possible outputs for a given batch from the N b batches; and
the coefficients of the tensors in the MPS model minimize the cost function sequentially over all the N b batches.
2 . The system of claim 1 , wherein the at least one processor is further configured to:
decompose the weight vector W using tensor network support vector regression to obtain the weight tensors.
3 . The system of claim 1 , wherein the at least one processor is configured to train the MPS model to minimize loss over all the N b batches by sweeping the MPS and optimizing the coefficients of the tensors until a convergence criterion is satisfied.
4 . The system of claim 3 , wherein the at least one processor is configured to optimize the coefficients of the tensors using one of: auto-differentiation and back propagation, density matrix renormalization group (DMRG) variational sweep, or tangent space variational optimization.
5 . The system of claim 1 , wherein the feature map Φ is represented by D sets of two-dimensional feature vectors, the MPS is represented by D wings of tensors, and each wing is associated with one set of two-dimensional feature vectors.
6 . The system of claim 5 , wherein the D extra tensors in the MPS model having a physical dimension of size M are central tensors in the MPS model.
7 . The system of claim 1 , wherein each fitting function is determined using tensor network support vector regression.
8 . The system of claim 1 , wherein each fitting function is determined using support vector regression (SVR) in combination with tensor network (TN) decompositions.
9 . The system of claim 1 , wherein the at least one processor is further configured to:
flatten the MPS model by cutting a virtual bond between neighboring tensors.
10 . The system of claim 1 , wherein the cost function is a log-cosh cost function.
11 . A computer-implemented method of applying non-linear regression on a set of N datapoints to get an estimate comprising:
receiving the set of N datapoints, each of the datapoints having associated input values;
separating the set of N datapoints into N b batches, each of the N b batches having M=N/N b datapoints;
receiving a family of fitting functions, each function comprising a dot product of a feature map Φ with a weight vector W, where the feature map Φ is local and represented by two-dimensional feature vectors; and
minimizing a cost function for each of the N b batches by training a matrix product state (MPS) model to minimize loss over all the N b batches, the MPS model providing an MPS representation of the weight vector W and comprising M+D tensors,
wherein D corresponds to a dimension of a target function, and
wherein each of the tensors have respective coefficients, such that:
all tensors except D correspond to one datapoint i in each of the N b batches;
D extra tensors in the MPS have a physical dimension of size M corresponding to a number of possible outputs for a given batch from the N b batches; and
the coefficients of the tensors in the MPS minimize the cost function sequentially over all the N b batches.
12 . The method of claim 11 , further comprising decomposing the weight vector W using tensor network support vector regression to obtain the weight tensors.
13 . The method of claim 11 , wherein training the MPS model to minimize loss over all the Ny batches comprises sweeping the MPS and optimizing the coefficients of the tensors until a convergence criterion is satisfied.
14 . The method of claim 13 , wherein the coefficients of the tensors are optimized using one of: auto-differentiation and back propagation, density matrix renormalization group (DMRG) variational sweep, or tangent space variational optimization.
15 . The method of claim 11 , wherein the feature map Φ is represented by D sets of two-dimensional feature vectors, the MPS is represented by D wings of tensors, and each wing is associated with one set of two-dimensional feature vectors.
16 . The method of claim 15 , wherein the D extra tensors in the MPS having a physical dimension of size M are central tensors in the MPS model.
17 . The method of claim 11 , wherein each fitting function is determined using tensor network support vector regression.
18 . The method of claim 11 , wherein each fitting function is determined using support vector regression (SVR) in combination with tensor network (TN) decompositions.
19 . The method of claim 11 , further comprising flattening the MPS model by cutting a virtual bond between neighboring tensors.
20 . The method of claim 11 , wherein the cost function is a log-cosh cost function.