Combining multiple clusterings by soft correspondence
View Patent ↗Combining multiple clusterings arises in various important data mining scenarios. However, finding a consensus clustering from multiple clusterings is a challenging task because there is no explicit correspondence between the classes from different clusterings. Provided is a framework based on soft correspondence to directly address the correspondence problem in combining multiple clusterings. Under this framework, an algorithm iteratively computes the consensus clustering and correspondence matrices using multiplicative updating rules. This algorithm provides a final consensus clustering as well as correspondence matrices that gives intuitive interpretation of the relations between the consensus clustering and each clustering from clustering ensembles. Extensive experimental evaluations demonstrate the effectiveness and potential of this framework as well as the algorithm for discovering a consensus clustering from multiple clusterings.
1. A method of determining a consensus clustering and correspondence matrices from a plurality of membership matrices, comprising:
iteratively computing, with an automated processor, from r membership matrices M (1) ∈ n×k1 , . . . , M (r) ∈ n×kr , k∈ + , the consensus clustering represented by M∈ n×k and the r correspondence matrices S (1) ∈ k×k , . . . , S (h) ∈ kh×k ,
by minimization of an objective function comprising a consensus function f,
updating S (h) using update rule:
S
(
h
)
←
S
(
h
)
-
Θ
⊙
(
∂
f
∂
S
(
h
)
)
until predetermined convergence conditions are achieved,
wherein ⊙ denotes the Hadamard product of two matrices,
Θ is a matrix of step size parameters, and
updating M using update rule:
M
←
M
=
1
r
∑
h
=
1
r
M
(
h
)
S
(
h
)
until predetermined convergence conditions are achieved; and
storing the computed consensus clustering and the correspondence matrices in a memory.
2. The method according to claim 1 , wherein the objective function comprises a Euclidean distance.
3. The method according to claim 1 , wherein the objective function comprises a term added to a consensus function in order to enforce a matrix sparseness constraint.
4. The method according to claim 1 , wherein the objective function comprises a penalty term added to a consensus function in order to enforce an external
∑
j
S
ij
(
h
)
=
1
constraint.
5. The method according to claim 1 , wherein the objective function comprises a consensus function term, a matrix sparseness constraint enforcement term, and a penalty term in order to enforce an external
∑
j
S
ij
(
h
)
=
1
constraint.
6. The method according to claim 1 , wherein
Θ
=
S
(
h
)
2
D
,
and
D
=
(
M
(
h
)
)
T
M
(
h
)
S
(
h
)
-
α
S
(
h
)
+
α
k
h
1
k
h
k
h
S
(
h
)
+
β
kS
(
h
)
1
kk
,
such that the updating rule for S (h) is:
S
(
h
)
←
S
(
h
)
⊙
(
M
(
h
)
)
T
M
+
β
k
1
k
h
k
D
+
ɛ
,
where:
α is a constant ≧0, selected to enforce a column-sparseness constraint, and
β is a scaling constant ≧0, for scaling a penalty term added to a consensus function f(M,S (1) ,S (2) , . . . , S (r) ) to deal with an external constraint
∑
j
S
ij
(
h
)
=
1
efficiently,
k denotes the number of clusters,
1 khkh denotes a k h -by-k h matrix of 1s, and
ε is a very small positive number used to avoid dividing by 0.
7. The method according to claim 1 , wherein
S
=
arg
min
Y
M
-
M
(
0
)
Y
2
,
where
·
denotes Frobenius matrix norm, and the objective function comprises a function of:
the consensus function f,
a term
α
∑
h
=
1
r
S
(
h
)
-
(
1
/
k
h
)
1
khkh
S
(
h
)
2
to enforce the column sparseness constraint, and
a penalty term
β
∑
h
=
1
r
S
(
h
)
1
kk
-
1
khk
2
added to the consensus function to implement a constraint
∑
j
S
ij
(
h
)
=
1
,
where α and β are constants ≧0, k denotes the number of clusters, and 1 khkh denotes a k h -by-k h matrix of 1s.
8. The method according to claim 1 , wherein the membership matrices comprise clusterings of content anchor text, uniform resource locators (URLs), and hyperlinks of hyperlinked documents.
9. A non-transitory computer readable medium for controlling an automated system to iteratively compute from r membership matrices M (1) ∈ n×k1 , . . . , M (r) ∈ n×kr , and k∈ + , the consensus clustering represented by M∈ n×k and the r correspondence matrices S (1) ∈ k1×k , . . . , S (h) ∈ kh×k ,
by minimization of an objective function comprising a consensus function f,
updating S (h) using update rule:
S
(
h
)
←
S
(
h
)
-
Θ
⊙
(
∂
f
∂
S
(
h
)
)
until predetermined convergence conditions are achieved, wherein ⊙ denotes the Hadamard product of two matrices, Θ is a matrix of step size parameters, and
updating M using update rule:
M
←
M
=
1
r
∑
h
=
1
r
M
(
h
)
S
(
h
)
until predetermined convergence conditions are achieved; and
storing the computed consensus clustering and the correspondence matrices in a memory.
10. The non-transitory computer readable medium according to claim 9 , wherein the objective function comprises a Euclidean distance, and the objective function comprises a consensus function term, a matrix sparseness constraint enforcement term, and a penalty term in order to enforce an external
∑
j
S
ij
(
h
)
=
1
constraint.
11. The non-transitory computer readable medium according to claim 10 , wherein
Θ
=
S
(
h
)
2
D
,
and
D
=
(
M
(
h
)
)
T
M
(
h
)
S
(
h
)
-
α
S
(
h
)
+
α
k
h
1
k
h
k
h
S
(
h
)
+
β
kS
(
h
)
1
kk
,
such that the updating rule for S (h) is:
S
(
h
)
←
S
(
h
)
⊙
(
M
(
h
)
)
T
M
+
β
k
1
k
h
k
D
+
ɛ
,
where:
α is a constant ≧0, selected to enforce a column-sparseness constraint, and
β is a scaling constant ≧0, for scaling a penalty term added to a consensus function f(M,S (1) ,S (2) , . . . , S (r) to deal with an external constraint
∑
j
S
ij
(
h
)
=
1
efficiently,
k denotes the number of clusters,
1 khkh denotes a k h -by-k h matrix of 1s, and
ε is a very small positive number used to avoid dividing by 0.
12. The non-transitory computer readable medium according to claim 9 , wherein
S
=
arg
min
Y
M
-
M
(
0
)
Y
2
,
where ∥·∥ denotes Frobenius matrix norm, and the objective function comprises:
the consensus function f;
a term
α
∑
h
=
1
r
S
(
h
)
-
(
1
/
k
h
)
1
khkh
S
(
h
)
2
added to the consensus function to enforce a column sparseness constraint; and
a penalty term
β
∑
h
=
1
r
S
(
h
)
1
kk
-
1
khk
2
added to the consensus function to meet external constraint
∑
j
S
ij
(
h
)
=
1
,
where α and β are constants ≧0, k denotes the number of clusters, and 1 khkh denotes a k h -by-k h matrix of 1s.
13. The non-transitory computer readable medium according to claim 9 , wherein the membership matrices comprise clusterings of content anchor text, uniform resource locators (URLs), and hyperlinks of hyperlinked documents.
14. The non-transitory computer readable medium according to claim 9 , wherein the automated system computes an algorithm comprising:
1: Initialize M,S (1) , ... , S (r) ,
2: while convergence criterion of M is not satisfied do
3: for h = 1 to r do
4: while convergence criterion of S(h) is not satisfied
do
5: S (h) ← S (h) ⊙ ((M (h) ) T M + βk1 khk )/(D+ε)
6: end while
7: end for
8:
M
=
1
/
r
∑
h
=
1
r
M
(
h
)
S
(
h
)
9: end while
where:
α is a constant ≧0, selected to enforce a column-sparseness constraint, and
β is a scaling constant ≧0, for scaling a penalty term added to a consensus function f(M,S (1) ,S (2) , . . . , S (r) ) to deal with an external constraint
∑
j
S
ij
(
h
)
=
1
efficiently,
k denotes the number of clusters,
1 khkh denotes a k h -by-k h matrix of 1s, and
ε is a very small positive number used to avoid dividing by 0.
15. A method for combining multiple clusterings from a plurality of membership matrices into a consensus clustering and correspondence matrices, comprising:
iteratively computing the consensus clustering and correspondence matrices with an automated processor, from the plurality of membership matrices by minimization of an objective function comprising at least an consensus function, a term to enforce a column sparseness constraint, and a term to enforce a matrix row sum constraint, until predetermined convergence conditions are achieved, until predetermined convergence conditions are satisfied; and
storing the computed consensus clustering and the correspondence matrices in a memory.
16. The method according to claim 15 , wherein the consensus function comprises a Euclidean distance.
17. The method according to claim 15 , wherein said iteratively computing comprises:
iteratively computing from r membership matrices M (1) ∈ n×k1 , . . . , M (r) ∈ n×kr , and k∈ + , the consensus clustering represented by M∈ n×k and the r correspondence matrices S (1) ∈ k1×k , . . . , S (h) ∈ kh×k ,
by minimization of an objective function comprising a consensus function f,
updating S (h) using update rule:
S
(
h
)
←
S
(
h
)
-
Θ
⊙
(
∂
f
∂
S
(
h
)
)
until predetermined convergence conditions are achieved, wherein ⊙ denotes the Hadamard product of two matrices, Θ is a matrix of step size parameters, and
updating M using update rule:
M
←
M
=
1
r
∑
h
=
1
r
M
(
h
)
S
(
h
)
until predetermined convergence conditions are achieved.
18. The method according to claim 17 , wherein
S
=
arg
min
Y
M
-
M
(
0
)
Y
2
,
where ∥·∥ denotes Frobenius matrix norm, and the objective function comprises a function of:
the consensus function f,
a term
α
∑
h
=
1
r
S
(
h
)
-
(
1
/
k
h
)
1
khkh
S
(
h
)
2
to enforce the column sparseness constraint, and
a penalty term
β
∑
h
=
1
r
S
(
h
)
1
kk
-
1
khk
2
added to the consensus function to implement a constraint
∑
j
S
ij
(
h
)
=
1
,
where α and β are constants ≧0, k denotes the number of clusters, and 1 khkh denotes a k h -by-k h matrix of 1s.
19. The method according to claim 15 , wherein:
Θ
=
S
(
h
)
2
D
,
and
D
=
(
M
(
h
)
)
T
M
(
h
)
S
(
h
)
-
α
S
(
h
)
+
α
k
h
1
k
h
k
h
S
(
h
)
+
β
kS
(
h
)
1
kk
,
such that the updating rule for S (h) is:
S
(
h
)
←
S
(
h
)
⊙
(
M
(
h
)
)
T
M
+
β
k
1
k
h
k
D
+
ɛ
,
where:
αis a constant ≧0, selected to enforce a column-sparseness constraint, and
β is a scaling constant ≧0, for scaling a penalty term added to a consensus function f(M,S (1) ,S (2) , . . . , S (r) ) to deal with an external constraint
∑
j
S
ij
(
h
)
=
1
efficiently,
k denotes the number of clusters,
1 khkh denotes a k h -by-k h matrix of 1s, and
ε is a very small positive number used to avoid dividing by 0.
20. The method according to claim 15 , wherein the membership matrices comprise clusterings of content anchor text, uniform resource locators (URLs), and hyperlinks of hyperlinked documents.