Methods for feature selection in a learning machine
View Patent ↗In a pre-processing step prior to training a learning machine, pre-processing includes reducing the quantity of features to be processed using feature selection methods selected from the group consisting of recursive feature elimination (RFE), minimizing the number of non-zero parameters of the system (l 0 -norm minimization), evaluation of cost function to identify a subset of features that are compatible with constraints imposed by the learning set, unbalanced correlation score and transductive feature selection. The features remaining after feature selection are then used to train a learning machine for purposes of pattern classification, regression, clustering and/or novelty detection.
1. A computer readable medium having stored thereon one or more sequences of instructions for causing one or more microprocessors to perform the steps for executing a kernel machine for classifying data in a large dataset into two or more classes and for selecting a subset of features within the dataset for processing in the kernel machine, the steps comprising:
executing a feature selection algorithm on the dataset to identify the subset of features, wherein the feature selection algorithm comprises an approximation of l 0 -norm minimization to find a smallest number of non-zero elements of a weight vector w, where w=(1, . . . , 1) and w k is given, by repeating the steps of calculating until convergence:
min
w
w
2
2
subject
to
:
y
i
(
〈
w
,
(
x
i
*
w
k
)
〉
+
b
)
≥
1
,
and w k+1 =w k *ŵ, where Ŵ is the solution of the previous problem;
using the kernel machine, processing the subset of features of the dataset to identify a pattern; and
generating an output to a printer or display device, the output comprising the identified pattern within the dataset.
2. The computer readable medium of claim 1 , wherein the kernel machine is a support vector machine.
3. The computer readable medium of claim 1 , wherein the dataset comprises a multi-label dataset, and further comprising calculating a label set size.
4. The computer readable medium of claim 3 , wherein the step of calculating label set size comprises minimizing a ranking loss.
5. The computer readable medium of claim 3 , wherein the dataset comprises gene expression data obtained from DNA micro-arrays.
6. The computer readable medium of claim 1 , further comprising mapping the dataset into feature space prior to executing the feature selection algorithm.
7. The computer readable medium of claim 6 , wherein the dataset comprises gene expression data obtained from DNA micro-arrays.
8. A computer readable medium having stored thereon one or more sequences of instructions for causing one or more microprocessors to perform the steps for selecting a subset of features from a plurality of features that describe attributes of the data points within a large dataset for processing in a kernel machine for recognition of patterns within the dataset, the steps comprising:
executing a feature selection algorithm on the dataset to identify a subset of features, wherein the algorithm comprises l 2 -AL0M; and
using the selected subset of features, analyzing the dataset to identify patterns therein.
9. The computer readable medium of claim 8 , wherein the kernel machine is a support vector machine.
10. The computer readable medium of claim 8 , wherein the dataset comprises a multilabel dataset, and further comprising the step of calculating a label set size.
11. The computer readable medium of claim 10 , wherein the step of calculating label set size comprises minimizing a ranking loss.
12. The computer readable medium of claim 8 , wherein the dataset comprises gene expression data obtained from DNA micro-arrays.
13. The computer readable medium of claim 8 , further comprising the step of mapping the dataset into feature space prior to executing the feature selection algorithm.
14. The computer readable medium of claim 9 , further comprising the step of applying the l 2 -AL0M algorithm to the support vectors (α 1 , . . . ,α l ) of the support vector machine to obtain a sparse-SVM with a minimum number of support vectors.
15. The computer readable medium of claim 14 , wherein the l 2 -AL0M algorithm as applied to the support vectors comprises finding
min
α
i
,
ξ
i
α
0
subject
to
:
y
i
(
∑
j
=
1
l
α
j
k
(
x
j
,
x
i
)
)
≥
1.
16. The computer readable medium of claim 8 , wherein the dataset comprises a large set of codebook vectors and the subset of features comprises a reduced codebook of vectors, and wherein the pattern comprises signal compression or decompression.
17. The computer readable medium of claim 2 , further comprising the step of applying the approximation of l 0 -norm minimization to the support vectors (α 1 , . . . ,α l ) of the support vector machine to obtain a sparse-SVM with a minimum number of support vectors.
18. The computer readable medium of claim 17 , wherein the approximation of l 0 - norm minimization as applied to the support vectors comprises finding
min
α
i
,
ξ
i
α
0
subject
to
:
y
i
(
∑
j
=
1
l
α
j
k
(
x
j
,
x
i
)
)
≥
1.
19. The computer readable medium of claim 1 , wherein the dataset comprises codebook vectors and the subset of features comprises a reduced codebook of vectors, and wherein the pattern comprises signal compression or decompression.