Method for classification of newly arrived multidimensional data points in dynamic big data sets
View Patent ↗A method for classification of a newly arrived multidimensional data point (MDP) in a dynamic data uses multi-scale extension (MSE). The multi-scale out-of-sample extension (OOSE) uses a coarse-to-fine hierarchy of the multi-scale decomposition of a Gaussian kernel that established the distances between MDPs in a training set to find the coordinates of newly arrived MDPs in an embedded space. A well-conditioned basis is first generated in a source matrix of MDPs. A single-scale out-of-sample extension (OOSE) is applied to the newly arrived MDP on the well-conditioned basis to provide coordinates of an approximate location of the newly arrived MDP in an embedded space. A multi-scale OOSE is then applied to the newly arrived MDP to provide improved coordinates of the newly arrived MDP location in the embedded space.
1. A method for classification of a newly arrived multidimensional data point (MDP) in a dynamic data set, comprising the steps of:
a) generating a well-conditioned basis in a source matrix of multidimensional data points;
b) applying a single-scale out-of-sample extension (OOSE) to the newly arrived MDP on the well-conditioned basis to provide coordinates of an approximate location of the newly arrived MDP in an embedded space; and
c) applying a multi-scale OOSE to the newly arrived MDP to provide improved coordinates of the newly arrived MDP location in the embedded space, thereby classifying the newly arrived MDP in relation to the source matrix multidimensional data points of the dynamic data set.
2. The method of claim 1 , wherein the step of generating a well-conditioned basis includes generating the well-conditioned basis using a randomized interpolative decomposition (RID) algorithm.
3. The method of claim 1 , wherein the source matrix associated with a Gaussian kernel matrix, and wherein the step of applying a single-scale OOSE includes decomposing the associated Gaussian kernel matrix in the single scale, thereby obtaining the coordinates of the approximate location of the newly arrived MDP in the embedded space.
4. The method of claim 3 , wherein the step of applying a multi-scale OOSE includes decomposing the associated Gaussian kernel matrix in a multi-scale, thereby obtaining the coordinates of the improved location of the newly arrived MDP in the embedded space.
5. The method of claim 2 , wherein the source matrix associated with a Gaussian kernel matrix, and wherein the step of applying a single-scale OOSE includes decomposing the associated Gaussian kernel matrix in the single scale, thereby obtaining the coordinates of the approximate location of the newly arrived MDP in the embedded space.
6. The method of claim 5 , wherein the step of applying a multi-scale OOSE includes decomposing the associated Gaussian kernel matrix in a multi-scale, thereby obtaining the coordinates of the improved location of the newly arrived MDP in the embedded space.
7. The method of claim 1 , wherein the step of applying a multi-scale OOSE to the newly arrived MDP includes applying the multi-scale OOSE to the newly arrived MDP on the well-conditioned basis.
8. The method of claim 7 , wherein the step of generating a well-conditioned basis includes generating the well-conditioned basis using a randomized interpolative decomposition (RID) algorithm.
9. The method of claim 7 , wherein the source matrix associated with a Gaussian kernel matrix, and wherein the step of applying a single-scale OOSE includes decomposing the associated Gaussian kernel matrix in the single scale, thereby obtaining the coordinates of the approximate location of the newly arrived MDP in the embedded space.
10. The method of claim 9 , wherein the step of applying a multi-scale OOSE includes decomposing the associated Gaussian kernel matrix in a multi-scale, thereby obtaining the coordinates of the improved location of the newly arrived MDP in the embedded space.
11. The method of claim 8 , wherein the source matrix associated with a Gaussian kernel matrix, and wherein the step of applying a single-scale OOSE includes decomposing the associated Gaussian kernel matrix in the single scale, thereby obtaining the coordinates of the approximate location of the newly arrived MDP in the embedded space.
12. The method of claim 11 , wherein the step of applying a multi-scale OOSE includes decomposing the associated Gaussian kernel matrix in a multi-scale, thereby obtaining the coordinates of the improved location of the newly arrived MDP in the embedded space.
13. The method of claim 1 , wherein the dataset includes a big data set.
14. The method of claim 1 , wherein the data set includes a network traffic dataset.
15. The method of claim 7 , wherein the dataset includes a big data set.
16. The method of claim 7 , wherein the data set includes a network traffic dataset.