Automatically building business intelligence models
The present disclosure relates to methods and systems that automatically predict a business intelligence model for tables of data provided as input. The methods and systems automatically generate a graph representing the business intelligence model and provide the graph as output. The graph provides a visual representation of the business intelligence model with nodes of the graph representing each input table and edges of the graph representing weighted edges joining pairs of tables together.
1 . A method, comprising:
receiving tables of data;
using a machine learning model to automatically predict a business intelligence model for the tables by predicting a probability of joinability of each pair of table columns in the tables, wherein the business intelligence model defines relationships between the data and the probability of joinability of a pair of table columns is used in creating edges of a graph;
outputting the graph for the business intelligence model using the probability of joinability, wherein the graph is constructed in a general snowflake structure using an equation that uses the probability of joinability in identifying nodes and edges of the graph and the nodes of the graph represent each input table of the tables of data and the edges of the graph represent weighted edges joining pairs of tables together;
using the machine learning model to perform an optimization leveraging graph properties and a general shape of business intelligence models to generate accurate predictions of join relationships between the tables in the business intelligence model, wherein the optimization enforces the snowflake structure and identifies and removes improper edges from the predicted probability of joinability in the graph; and
outputting the graph based on modifications from the optimization.
2 . The method of claim 1 , wherein the machine learning model is trained offline to predict the probability of joinability of each pair of tables in the tables of data.
3 . The method of claim 2 , wherein the graph further includes the probability of joinability presented on the edges of the graph.
4 . The method of claim 1 , further comprising:
performing an optimization of the graph using graph properties.
5 . The method of claim 4 , wherein the graph properties enforce a structure on the graph to perform the optimization of the graph.
6 . The method of claim 5 , wherein the graph properties include a minimum cost arborescence, an edge maximizing schema, or a cardinality constraint.
7 . The method of claim 4 , wherein the optimization includes adding edges to the graph.
8 . The method of claim 4 , wherein the optimization includes removing edges from the graph.
9 . The method of claim 1 , further comprising:
performing a recall mode optimization on the graph identifying missing joins from the predicted probability of joins in the graph; and
adding the missing joins as edges to the graph.
10 . A device, comprising:
a processor;
memory in electronic communication with the processor; and
instructions stored in the memory, the instructions being executable by the processor to:
receive tables of data;
use a machine learning model to automatically predict a business intelligence model for the tables by predicting a probability of joinability of each pair of table columns in the tables, wherein the business intelligence model defines relationships between the data and the probability of joinability of a pair of table columns is used in creating edges of a graph;
output the graph for the business intelligence model using the probability of joinability, wherein the graph is constructed in a general snowflake structure using an equation that uses the probability of joinability in identifying nodes and edges of the graph and the nodes of the graph represent each input table of the tables of data and edges of the graph represent weighted edges joining pairs of tables together;
use the machine learning model to perform an optimization leveraging graph properties and a general shape of business intelligence models to generate accurate predictions of join relationships between the tables in the business intelligence model, wherein the optimization enforces the snowflake structure and identifies and removes improper edges from the predicted probability of joinability in the graph; and
output the graph based on modifications from the optimization.
11 . The device of claim 10 , wherein the machine learning model is trained offline to predict a probability of joinability of each pair of tables in the tables of data and the probability of joinability is used in creating the edges of the graph.
12 . The device of claim 11 , wherein the graph further includes the probability of joinability presented on the edges of the graph.
13 . The device of claim 10 , wherein the instructions are further executable by the processor to perform an optimization of the graph using graph properties.
14 . The device of claim 13 , wherein the graph properties enforce a structure on the graph to perform the optimization of the graph.
15 . The device of claim 14 , wherein the graph properties include a minimum cost arborescence, an edge maximizing schema, or a cardinality constraint.
16 . The device of claim 13 , wherein the instructions are further executable by the processor to perform the optimization on the graph by adding edges to the graph or removing edges from the graph.