Techniques for building or querying graph model of data stored in a common data format
Described are examples for creating a graph model for data obtained from multiple data sources and stored in a common data format. A selection of columns of data stored in the common data format that are to be associated with one another in the graph model can be obtained. Based on the selection, a graph model can be created including metadata that defines a list of vertices corresponding to at least a portion of the columns and a list of edges between two or more of the vertices identifying a relationship between corresponding columns. A graph query or graph algorithm can be performed over the graph model of the data as stored in the common data format without requiring copying of the data to a separate graph database.
1 . A device for creating a graph model for data obtained from multiple data sources and stored in a common data format, comprising:
one or more memories storing instructions; and
one or more processors coupled to the one or more memories and configured to execute the instructions to:
obtain a selection of columns of data stored in a first data store in the common data format that are to be associated with one another in the graph model;
create, based on the selection and without copying the data from the columns to a second data store that is different from the first data store, a graph model including metadata that defines a list of vertices corresponding to at least a portion of the columns and a list of edges between two or more vertices in the list of vertices identifying a relationship between corresponding columns;
receive a query request for a query to execute over the data stored in the first data store;
populate, based on the graph model and the query request, a graph index, stored in the one or more memories, with:
first data in a first column, indicated in the query request, that corresponds to a first vertex in the list of vertices; and
second data in a second column, indicated in the query request, that corresponds to a second vertex in the list of vertices,
wherein populating the graph index with the first data and the second data is based on one of an edge in the list of edges that is between the first vertex and the second vertex; and
perform the query over the graph model of the data as stored in the common data format and based on the graph index.
2 . The device of claim 1 , wherein the one or more processors are configured to execute the instructions to create the graph model including specifying, in metadata defining the graph model and based on the selection, at least one of the portion of the columns as a property for at least one vertex in the list of vertices or for at least one edge in the list of edges.
3 . The device of claim 1 , wherein the one or more processors are configured to execute the instructions to populate, based on the graph model, the graph index stored in the one or more memories with third data in a third column based on the third column indicated, in the graph model, as a property for the first vertex.
4 . The device of claim 1 , wherein the one or more processors are configured to execute the instructions to update the graph index based on additional rows of data added to the data stored in the common data format and having at least the first column or the second column indicated in the graph model.
5 . The device of claim 1 , wherein the one or more processors are configured to execute the instructions to populate the graph index based on receiving the query indicating at least the first column in the query.
6 . The device of claim 1 , wherein the one or more processors are configured to execute the instructions to return, to a graphical user interface (GUI), results of the query.
7 . A computer-implemented method of creating a graph model for data obtained from multiple data sources and stored in a common data format, comprising:
creating, based on a selection of columns of data stored in a first data store in the common data format that are to be associated with one another and without copying the data from the columns to a second data store that is different from the first data store, a graph model including metadata that defines a list of vertices corresponding to at least a portion of the columns and a list of edges between two or more vertices in the list of vertices identifying a relationship between corresponding columns;
receive a query request for a query to execute over the data stored in the first data store;
populate, based on the graph model and the query request, a graph index, stored in one or more memories, with:
first data in a first column, indicated in the query request, that corresponds to a first vertex in the list of vertices; and
second data in a second column, indicated in the query request, that corresponds to a second vertex in the list of vertices,
wherein populating the graph index with the first data and the second data is based on one of an edge in the list of edges that is between the first vertex and the second vertex; and
performing the query over the graph model of the data as stored in the common data format and based on the graph index.
8 . The computer-implemented method of claim 7 , wherein creating the graph model includes specifying, in metadata that defines the graph model and based on the selection, at least one of the portion of the columns as a property for at least one vertex in the list of vertices or for at least one edge in the list of edges.
9 . The computer-implemented method of claim 7 , further comprising populating, based on the graph model, the graph index in memory with third data in a third column based on the third column indicated, in the graph model, as a property for the first vertex.
10 . The computer-implemented method of claim 7 , further comprising updating the graph index as a batch process and based on additional rows of data added to the data stored in the common data format and having at least the first column or the second column indicated in the graph model.
11 . The computer-implemented method of claim 7 , wherein populating the graph index is based on receiving the query indicating at least the first column in the query.
12 . The computer-implemented method of claim 7 , further comprising returning, via a graphical user interface (GUI), results of the query.
13 . One or more non-transitory computer-readable media storing instructions thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations for creating a graph model for data obtained from multiple data sources and stored in a common data format, comprising:
obtaining a selection of columns of data stored in a first data store in the common data format that are to be associated with one another in the graph model;
creating, based on the selection and without copying the data from the columns to a second data store that is different from the first data store, a graph model including metadata that defines a list of vertices corresponding to at least a portion of the columns and a list of edges between two or more vertices in the list of vertices identifying a relationship between corresponding columns;
receive a query request for a query to execute over the data stored in the first data store;
populate, based on the graph model and the query request, a graph index, stored in one or more memories, with:
first data in a first column, indicated in the query request, that corresponds to a first vertex in the list of vertices; and
second data in a second column, indicated in the query request, that corresponds to a second vertex in the list of vertices,
wherein populating the graph index with the first data and the second data is based on one of an edge in the list of edges that is between the first vertex and the second vertex; and
performing the query over the graph model of the data as stored in the common data format and based on the graph index.
14 . The one or more non-transitory computer-readable media of claim 13 , wherein creating the graph model includes specifying, in metadata defining the graph model and based on the selection, at least one of the portion of the columns as a property for at least one vertex in the list of vertices or for at least one edge in the list of edges.
15 . The one or more non-transitory computer-readable media of claim 13 , the operations further comprising populating, based on the graph model, the graph index in memory with third data in a third column based on the third column indicated, in the graph model, as a property for the first vertex.