Singular value decompositions
In one example in accordance with the present disclosure, a system comprises a computing node. The computing node comprises: a memory, and a processor to: execute a database in the memory, and invoke, with the database, singular value decomposition (SVD) on a data set. To invoke SVD, the processor may sparsify, with the database, the data set to produce a sparse data set, iteratively decompose, with the database, the data set to produce a set of eigenvalues, solve, with the database a linear system to produce a set of eigenvectors, and multiply, with the database, the eigenvectors with the data set to produce a data set of reduced dimension.
1. A method for invoking a singular value decomposition (SVD) on a data set by a database management system (DBMS), the method comprising:
extracting, by a processor of the DBMS, information from the data set stored in a database of the DBMS;
creating, by the processor of the DBMS, a matrix based on the data set, including:
in response to a determination that a size of a feature of the data set is less than a threshold value, creating the matrix locally by the processor of the DBMS, or
in response to a determination that the size of the feature of the data set is greater than the threshold value, creating the matrix, by the processor of the DBMS, in a distributed fashion across a plurality of nodes;
generating, by the processor of the DBMS, a sparse matrix from the created matrix;
determining, by the processor of the DBMS, eigenvectors from the sparse matrix; and
multiplying, by the processor of the DBMS, the eigenvectors to the data set to generate a new data set of reduced dimension.
2. The method of claim 1 , further comprising:
performing, with the DBMS, a QR decomposition on the sparse matrix to determine eigenvalues.
3. The method of claim 1 , wherein the size of the feature of the data set includes a size of a column of a table of the data set.
4. The method of claim 1 , wherein creating the matrix in the distributed fashion across the plurality of nodes comprises creating the matrix using a structured query language (SQL) statement.
5. The method of claim 2 , wherein creating the matrix locally by the processor of the DBMS comprises:
retrieving a binary column of the data set using a user-defined transform function; and
transforming the binary column of the data set into the matrix.
6. The method of claim 2 , wherein performing the QR decomposition on the sparse matrix comprises:
iteratively performing, with SQL statements executed by the DBMS, the QR decomposition on the sparse matrix.
7. The method of claim 1 , wherein creating the matrix locally by the processor of the DBMS comprises:
in response to a determination that a percentage of features for an eigenvalue determination is less than a threshold percentage, creating the matrix locally.
8. The method of claim 1 , wherein creating the matrix in the distributed fashion across the plurality of nodes comprises:
in response to a determination that a percentage of features for an eigenvalue determination is greater than a threshold percentage, creating the matrix, by the processor of the DBMS, in the distributed fashion across the plurality of nodes.
9. A database management system (DBMS) comprising a computing node, the computing node comprising:
a processor; and
a memory storing instructions that when executed cause the processor to:
extract information of a data set stored in a database of the DBMS;
create a matrix based on the data set, including causing the processor to:
in response to a determination that a size of a feature of the data set is less than a threshold value, create the matrix locally by the processor of the DBMS, or
in response to a determination that the size of the feature of the data set is greater than the threshold value, create the matrix in a distributed fashion across a plurality of nodes;
sparsify, with the DBMS, the created matrix to produce a sparse matrix;
produce a set of eigenvectors from the sparse matrix; and
multiply, with the DBMS, the set of eigenvectors with the data set to produce a new data set of a reduced dimension.
10. The DBMS of claim 9 , wherein, to create the matrix locally, the instructions are executable to cause the processor to:
retrieve a binary column of the data set using a user-defined transform function; and
transform the binary column of the data set into the matrix.
11. The DBMS of claim 9 , wherein, to create the matrix in the distributed fashion across the plurality of nodes, the instructions are executable to cause the processor to: create the matrix in the distributed fashion across the plurality of nodes using a structured query language (SQL) statement.
12. The DBMS of claim 9 , wherein the instructions are executable to cause the processor to: iteratively perform a QR decomposition on the sparse matrix to determine eigenvalues and the set of eigenvectors.
13. The DBMS of claim 9 , wherein the size of the feature of the data set includes a percentage of features for which eigenvalues are determined, and the threshold value is a threshold percentage.
14. The DBMS of claim 9 , wherein the size of the feature of the data set includes a size of a column of a table of the data set.
15. A non-transitory machine-readable storage medium of a database management system (DBMS) storing instructions that, when executed, cause a processor to:
extract information of a data set stored in a database of the DBMS;
create a matrix based on the data set, including causing the processor to:
in response to a determination that a size of a feature of the data set is less than a threshold value, create the matrix locally by the processor of the DBMS, or
in response to a determination that the size of the feature of the data set is greater than the threshold value, create the matrix in a distributed fashion across a plurality of nodes;
sparsify, with the DBMS, the created matrix to produce a sparse matrix;
generate a set of eigenvectors from the sparse matrix; and
multiply, with the DBMS, the set of eigenvectors with the data set to produce a new data set of a reduced dimension.
16. The non-transitory machine-readable storage medium of claim 15 , wherein the instructions are executable to cause the processor to:
iteratively perform a QR decomposition on the sparse matrix to determine eigenvalues and the set of eigenvectors.
17. The non-transitory machine-readable storage medium of claim 16 , wherein the size of the feature of the data set includes a percentage of features for which eigenvalues are determined, and the threshold value is a threshold percentage.
18. The non-transitory machine-readable storage medium of claim 15 , wherein, to create the matrix in the distributed fashion across the plurality of nodes, the instructions are executable to cause the processor to: invoke the SVD using:
create the matrix in the distributed fashion across the plurality of nodes using a structured query language (SQL) statement.
19. The non-transitory machine-readable storage medium of claim 15 , wherein the size of the feature of the data set includes a size of a column of a table of the data set.
20. The non-transitory machine-readable storage medium of claim 15 , wherein, to create the matrix locally, the instructions are executable to cause the processor to:
retrieve a binary column of the data set using a user-defined transform function; and
transform the binary column of the data set into the matrix.