Fast distributed nonnegative matrix factorization and completion for big data analytics
View Patent ↗Systems and methods are disclosed for operating a machine, by receiving training data from one or more sensors; training a machine learning module with the training data by: partitioning a data matrix into smaller submatrices to process in parallel and optimized for each processing node; for each submatrix, performing a greedy search for rank-one solutions; using alternating direction method of multipliers (ADMM) to ensure consistency over different data blocks; and controlling one or more actuators using live data and the learned module during operation.
1. A method for operating a machine, comprising:
receiving training data from one or more sensors;
training a machine learning module with the training data by:
partitioning a large-scale data matrix with a first rank into smaller submatrices;
for each submatrix, greedily searching for rank-one solutions of Nonnegative matrix factorization/completion (NMFC) based upon alternating direction method of multipliers (ADMM);
concatenating the sub-matrices to form low rank matrix factors, wherein the low rank is less than the first rank;
determining solutions of each submatrix and concatenating to form a final solution;
determining:
{
u
(
i
)
k
←
[
X
^
(
i
)
(
v
(
i
)
k
)
T
+
ρ
a
(
i
)
k
-
p
]
[
v
(
i
)
k
(
v
(
i
)
k
)
T
+
ρ
]
v
(
i
)
k
←
[
u
(
i
)
k
T
X
^
(
i
)
+
ρ
b
(
i
)
k
+
∑
j
≠
i
ρ
v
(
j
)
k
-
q
-
(
c
-
1
)
s
]
[
(
u
(
i
)
k
)
T
u
(
i
)
k
+
ρ
]
a
(
i
)
k
←
P
+
(
u
(
i
)
k
+
p
/
ρ
)
b
(
i
)
k
←
P
+
(
v
(
i
)
k
+
q
/
ρ
)
p
←
p
+
ρ
(
u
(
i
)
k
-
a
(
i
)
k
)
q
←
q
+
ρ
(
v
(
i
)
k
-
b
(
i
)
k
)
s
←
s
+
ρ
c
-
1
∑
j
≠
i
(
v
(
i
)
k
-
v
(
j
)
k
)
where X is a partially observed data matrix of n samples and d features; i, j, k are indices, a and b are auxiliary variables, c is a constant, p, q, and T are Lagrangian multipliers, ρ is a penalty operator, and U and V are nonnegative low rank matrix factors, and s is a closed form solution; and
controlling one or more actuators using live data and the learned module during operation.
2. The method of claim 1 , wherein the partitioning comprises selecting using one of random, rank-one singular value decomposition (SVD), and rank-one ADMM.
3. The method of claim 1 , comprising solving computation complexity as linearly proportional to a matrix rank.
4. The method of claim 1 , wherein for each submatrix X(i) with i as an index of submatrices, using rank-one ADMM to optimize NMFC over a node.
5. The method of claim 1 , comprising uniformly selecting a number of samples for a node.
6. The method of claim 1 , comprising performing singular value decomposition by projecting an eigenvector over a nonnegative set, and then conducting k-means clustering over the eigenvector.
7. The method of claim 1 , comprising performing ADMM for NMFC by conducting k-means clustering over an eigenvector.
8. The method of claim 1 , comprising providing a warm start for a second rank optimization.
9. The method of claim 1 , wherein for rank-one ADMM, comprising performing a greedy search for rank-one solutions based upon ADMM.
10. The method of claim 1 , comprising performing SVD by projecting an eigenvector over a nonnegative set, and then conducting k-means clustering over the eigenvector.
11. The method of claim 1 , comprising performing ADMM for Nonnegative matrix factorization/completion (NMFC) by conducting k-means clustering over an eigenvector.
12. The method of claim 1 , comprising providing a warm start for a second rank optimization.
13. The method of claim 1 , wherein for rank-one ADMM, comprising performing a greedy search for rank-one solutions based upon ADMM.
14. A method for operating a machine, comprising
receiving training data from one or more sensors;
training a machine learning module with the training data by:
partitioning a data matrix into smaller submatrices to process in parallel for each processing node;
for each submatrix, performing a greedy search for rank-one solutions;
using alternating direction method of multipliers (ADMM) to ensure consistency over different data blocks;
determining:
{
u
(
i
)
k
←
[
X
^
(
i
)
(
v
(
i
)
k
)
T
+
ρ
a
(
i
)
k
-
p
]
[
v
(
i
)
k
(
v
(
i
)
k
)
T
+
ρ
]
v
(
i
)
k
←
[
u
(
i
)
k
T
X
^
(
i
)
+
ρ
b
(
i
)
k
+
∑
j
≠
i
ρ
v
(
j
)
k
-
q
-
(
c
-
1
)
s
]
[
(
u
(
i
)
k
)
T
u
(
i
)
k
+
ρ
]
a
(
i
)
k
←
P
+
(
u
(
i
)
k
+
p
/
ρ
)
b
(
i
)
k
←
P
+
(
v
(
i
)
k
+
q
/
ρ
)
p
←
p
+
ρ
(
u
(
i
)
k
-
a
(
i
)
k
)
q
←
q
+
ρ
(
v
(
i
)
k
-
b
(
i
)
k
)
s
←
s
+
ρ
c
-
1
∑
j
≠
i
(
v
(
i
)
k
-
v
(
j
)
k
)
where X is a partially observed data matrix of n samples and d features; i, j, k are indices, a and b are auxiliary variables, c is a constant, p, q, and T are Lagrangian multipliers, ρ is a penalty operator, and U and V are nonnegative low rank matrix factors, and s is a closed form solution; and
controlling one or more actuators using live data and the learning module during operation.
15. The method of claim 14 , comprising partitioning the data matrix into smaller submatrices using one of: random, rank-one SVD, and rank-one ADMM technique.
16. The method of claim 14 , wherein for each submatrix X(i) with i as an index of submatrices, using rank-one ADMM to optimize Nonnegative matrix factorization/completion (NMFC) over a node.
17. The method of claim 14 , comprising uniformly selecting a number of samples for a node.