METHODS AND SYSTEMS FOR IMPROVED AUTOMATED MACHINE LEARNING AND DATA ANALYSIS
The disclosed methods and systems automate the process of building machine learning models. A user interface receives a selection of a dataset for a machine learning experiment. An execution plan for the experiment is determined based on the selected dataset. The experiment is executed according to the execution plan to generate a plurality of machine learning models. The performance of the generated models is evaluated based on one or more performance metrics. A model is selected from the generated models based on the evaluation of the performance metrics. The selected model may be stored for future use.
1 . A method comprising:
receiving, based on a user selection, a dataset for a machine learning experiment;
determining, based on the selected dataset, an execution plan for the machine learning experiment;
generating, based on the execution plan, a plurality of machine learning models through automated model training;
causing, based on completion of the automated model training, metadata associated with the plurality of machine learning models to be stored in a database;
determining, based on performance metrics, a selected model from the plurality of machine learning models;
generating, based on the selected model, prediction results and explanation data comprising SHAP values;
causing, based on a user request for analysis, the metadata and prediction results to be loaded into an associative engine for in-memory processing; and
generating, based on the loaded data in the associative engine, an interactive dashboard comprising visualizations that update dynamically in response to user selections.
2 . The method of claim 1 , wherein the metadata comprises at least one of model performance metrics, feature importance data, hyperparameters, preprocessing steps, or training configurations.
3 . The method of claim 1 , wherein the explanation data comprises SHAP values calculated for each feature contribution to individual predictions.
4 . The method of claim 1 , wherein the interactive dashboard comprises at least one of confusion matrices, feature importance charts, prediction distribution visualizations, or what-if scenario analysis controls.
5 . The method of claim 1 , further comprising:
generating, based on user input through the interactive dashboard, modified scenario parameters; and
causing, based on the modified scenario parameters, updated predictions to be displayed in real-time.
6 . The method of claim 1 , wherein the associative engine processes user selections to filter the metadata and prediction results instantaneously without requiring server queries.
7 . The method of claim 1 , wherein the execution plan comprises selecting algorithms from at least one of linear-based algorithms, tree-based algorithms, neural networks, or ensemble methods.
8 . A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations comprising:
receiving, based on a user selection, a dataset for a machine learning experiment;
determining, based on the selected dataset, an execution plan for the machine learning experiment;
generating, based on the execution plan, a plurality of machine learning models through automated model training;
causing, based on completion of the automated model training, metadata associated with the plurality of machine learning models to be stored in a database;
determining, based on performance metrics, a selected model from the plurality of machine learning models;
generating, based on the selected model, prediction results and explanation data comprising SHAP values;
causing, based on a user request for analysis, the metadata and prediction results to be loaded into an associative engine for in-memory processing; and
generating, based on the loaded data in the associative engine, an interactive dashboard comprising visualizations that update dynamically in response to user selections.
9 . The non-transitory computer-readable medium of claim 8 , wherein the metadata comprises at least one of model performance metrics, feature importance data, hyperparameters, preprocessing steps, or training configurations.
10 . The non-transitory computer-readable medium of claim 8 , wherein the explanation data comprises SHAP values calculated for each feature contribution to individual predictions.
11 . The non-transitory computer-readable medium of claim 8 , wherein the interactive dashboard comprises at least one of confusion matrices, feature importance charts, prediction distribution visualizations, or what-if scenario analysis controls.
12 . The non-transitory computer-readable medium of claim 8 , wherein the operations further comprise:
generating, based on user input through the interactive dashboard, modified scenario parameters; and
causing, based on the modified scenario parameters, updated predictions to be displayed in real-time.
13 . The non-transitory computer-readable medium of claim 8 , wherein the associative engine processes user selections to filter the metadata and prediction results instantaneously without requiring server queries.
14 . An apparatus comprising:
a processor; and
a memory storing instructions that, when executed by the processor, cause the apparatus to:
receive, based on a user selection, a dataset for a machine learning experiment;
determine, based on the selected dataset, an execution plan for the machine learning experiment;
generate, based on the execution plan, a plurality of machine learning models through automated model training;
cause, based on completion of the automated model training, metadata associated with the plurality of machine learning models to be stored in a database;
determine, based on performance metrics, a selected model from the plurality of machine learning models;
generate, based on the selected model, prediction results and explanation data comprising SHAP values;
cause, based on a user request for analysis, the metadata and prediction results to be loaded into an associative engine for in-memory processing; and
generate, based on the loaded data in the associative engine, an interactive dashboard comprising visualizations that update dynamically in response to user selections.
15 . The apparatus of claim 14 , wherein the metadata comprises at least one of model performance metrics, feature importance data, hyperparameters, preprocessing steps, or training configurations.
16 . The apparatus of claim 14 , wherein the explanation data comprises SHAP values calculated for each feature contribution to individual predictions.
17 . The apparatus of claim 14 , wherein the interactive dashboard comprises at least one of confusion matrices, feature importance charts, prediction distribution visualizations, or what-if scenario analysis controls.
18 . The apparatus of claim 14 , wherein the instructions further cause the apparatus to:
generate, based on user input through the interactive dashboard, modified scenario parameters; and
cause, based on the modified scenario parameters, updated predictions to be displayed in real-time.
19 . The apparatus of claim 14 , wherein the associative engine processes user selections to filter the metadata and prediction results instantaneously without requiring server queries.
20 . The apparatus of claim 14 , wherein the execution plan comprises selecting algorithms from at least one of linear-based algorithms, tree-based algorithms, neural networks, or ensemble methods.