Method for performing in-database distributed advanced predictive analytics modeling via common queries
A system, method, and computer-readable medium for performing a distributed analytics operation. The distributed analytics operation uses interface technologies to de-couple an actual data storage technology from an implementation of distributed analytics. Such a distributed analytics operation obviates requirements to deploy specific computer code onto a data storage platform to specifically target that platform for distributed predictive analytics computations.
1. A computer-implementable method for performing in-database processing operations, comprising:
generating an in-database processing operation according to user-selected decision variables and a database source description table;
instantiating the in-database processing operation within a node of a workspace environment, the node of the workspace environment being located away from a storage system and having a user interface for communicating with a user, the in-database processing operation comprising a storage system-agnostic operation performable without deployment of a storage system-specific agent;
converting, at the node of the workspace environment, the instantiation of the in-database processing operation to at least one SQL statement;
providing the at least one SQL statement to the storage system, wherein the at least one SQL statement is executed in-database within the storage system, and wherein the storage system executes a query language for initiating and controlling the in-database execution of the at least SQL statement;
receiving, at the node of the workspace environment, results from the in-database execution of the at least one SQL statement, wherein the results comprise aggregate values; and
performing, at the node of the workspace environment, mathematical and statistical computations for a predictive analytics modeling operation, the mathematical and statistical computations selected from a plurality of predictive analytics model operation using the aggregate values.
2. The method of claim 1 , wherein the at least one SQL statement is distributed across a plurality of nodes of the storage system, and wherein the at least one SQL statement is executed in-database at each of the plurality of nodes of the storage system.
3. The method of claim 1 , wherein the predictive modeling operation relies on queries to the storage system.
4. The method of claim 3 , wherein the predictive modeling operation comprises data-processing intensive operations.
5. The method of claim 1 , wherein the node of the workspace environment determines parameters for predictive models based upon the aggregate values.
6. The method of claim 1 , wherein the node of the workspace environment determines parameters for predictive models based upon the aggregate values.
7. The method of claim 1 , wherein the node comprises a feature selection module, a multiple regression module, a correlations module, and a logistic regression module.
8. The method of claim 1 , wherein subsamples of results of the in-database execution of the at least one SQL statement are created at the storage system and are accessed by the node via queries, and wherein the method further comprises iteratively computing predictive models based on the results.
9. The method of claim 8 , wherein the predictive models comprise a recursive partitioning model, a support vector machine, and a neural network.
10. The method of claim 8 , wherein the subsamples comprise consecutive samples, and wherein the method further comprises computing coefficients for the predictive models.
11. The method of claim 10 , further comprising refining the coefficients until the iterative computations no longer lead to modification of the coefficients.
12. The method of claim 11 , further comprising providing a user interface to parameterize the iterative computations.
13. The method of claim 10 , further comprising refining the coefficients until the iterative computations no longer lead to modification of the coefficients such that less than all data in the storage system is processed.
14. A system for performing in-database processing operations, the system comprising:
a processor;
a data bus coupled to the processor; and
a memory comprising instructions executable by the processor and configured for:
generating an in-database processing operation according to user-selected decision variables and a database source description table;
instantiating the in-database processing operation within a node of a workspace environment, the node of the workspace environment being located away from a storage system and having a user interface for communicating with a user, the in-database processing operation comprising a storage system-agnostic operation performable without deployment of a storage system-specific agent;
converting, at the node of the workspace environment, the instantiation of the in-database processing operation to at least one SQL statement;
providing the at least one SQL statement to the storage system, wherein the at least one SQL statement is executed in-database within the storage system, and wherein the storage system executes a query language for initiating and controlling the in-database execution of the at least SQL statement;
receiving, at the node of the workspace environment, results from the in-database execution of the at least one SQL statement, wherein the results comprise aggregate values; and
performing, at the node of the workspace environment, mathematical and statistical computations for a predictive analytics modeling operation, the mathematical and statistical computations selected from a plurality of predictive analytics model operations using the aggregate values.
15. The system of claim 14 , wherein the at least one SQL statement is distributed across a plurality of nodes of the storage system, and wherein the at least one SQL statement is executed in-database at each of the plurality of nodes of the storage system.
16. The system of claim 14 , wherein the predictive modeling operation relies on queries to the storage system.
17. The system of claim 16 , wherein the predictive modeling operation comprises data-processing intensive operations.
18. A non-transitory, computer-readable storage medium embodying computer program code, the computer program code comprising computer executable instructions configured for:
generating an in-database processing operation according to user-selected decision variables and a database source description table;
instantiating the in-database processing operation within a node of a workspace environment, the node of the workspace environment being located away from a storage system and having a user interface for communicating with a user, the in-database processing operation comprising a storage system-agnostic operation performable without deployment of a storage system-specific agent;
converting, at the node of the workspace environment, the instantiation of the in-database processing operation to at least one SQL statement;
providing the at least one SQL statement to the storage system, wherein the at least one SQL statement is executed in-database within the storage system, and wherein the storage system executes a query language for initiating and controlling the In-database execution of the at least SQL statement;
receiving, at the node of the workspace environment, results from the in-database execution of the at least one SQL statement, wherein the results comprise aggregate values; and
performing, at the node of the workspace environment, mathematical and statistical computations for a predictive analytics modeling operation, the mathematical and statistical computations selected from a plurality of predictive analytics mod& operations using the aggregate values.