System and method for partitioning data based on authorization rules

1 . A method for partitioning data and answering a query, comprising:

initiating a plurality of data nodes that includes at least a first data node and a second data node, wherein the first data node corresponds to a first authorization rule, and the second data node corresponds to a second authorization rule, wherein the first authorization rule authorizes a first user identifier to access the first data node, and the second authorization rule authorizes a second user identifier to access the second data node; and

prior to receiving a user request for data as part of a query, initially configuring an extract, transform, and load (ETL) application service to:

extract data from a data source;

load a first data from the data source to a first dataset stored on the first data node, based on the first authorization rule that authorizes a requesting user to access the first data node; and

load a second data from the data source to a second dataset stored on the second data node based on the second authorization rule that authorizes a requesting user to access the second data node, wherein the first dataset and the second dataset are based on a shared data model;

establishing a plurality of ETL pipelines, each ETL pipeline corresponding to one of the plurality of data nodes, wherein data filtered through each of the ETL pipelines is loaded into a respective dataset stored on a one of the plurality of data nodes corresponding to the ETL pipeline;

receiving a request from a device for data from one of the first data node and the second data node, the request including one of the first user identifier and the second user identifier, the data being requested specified as a query, and an identifier of the one of the first data node and the second data node storing the requested data;

determining whether the requesting device is associated with one of the first user identifier and the second user identifier that is authorized to access the one of the first data node and the second data node storing the requested data;

when the requesting device is associated with one of the first user identifier and the second user identifier that is authorized to access the one of the first data node and the second data node storing the requested data, executing the query on at least the one of the first data node and the second data node storing the requested data; and

transmitting results of the query to the requesting device.

2 . The method of claim 1 , wherein the data source is a first data source of a plurality of data sources, and wherein establishing the plurality of ETL pipelines further comprises:

extracting data from the plurality of data sources; and

loading the extracted data to the dataset stored on the first data node based on the first authorization rule.

3 . The method of claim 1 , wherein each of the first authorization rule and the second authorization rule further authorizes a respective user identifier to access at least one location of data.

4 . The method of claim 3 , wherein each of the first authorization rule and the second authorization rule each defines a respective key, wherein the at least one location of data authorized by each of the first authorization rule and the second authorization rule stores the respective key defined in the corresponding authorization rule.

5 . The method of claim 1 , further comprising:

generating a plurality of user groups, wherein each of the plurality of user groups includes at least one user identifier corresponding to one of the authorization rules.

6 . The method of claim 1 , wherein a group of data nodes each correspond to the first authorization rule.

7 . The method of claim 1 , further comprising:

executing the query on both of the first data node and the second data when the received one of the first user identifier and the second user identifier is authorized to access both of the first data node and the second data.

8 . The method of claim 1 , wherein initiating a first data node includes cause an orchestrator to initiate a new machine in a cloud computing environment.

9 . A non-transitory computer readable medium having stored thereon instructions for causing a processing circuitry to execute a process for partitioning data and answering a query, the process comprising:

initiating a plurality of data nodes that includes at least a first data node and a second data node, wherein the first data node corresponds to a first authorization rule, and the second data node corresponds to a second authorization rule, wherein the first authorization rule authorizes a first user identifier to access the first data node, and the second authorization rule authorizes a second user identifier to access the second data node; and

prior to receiving a user request for data as part of a query, initially configuring an extract, transform, and load (ETL) application service to:

extract data from a data source;

load a first data from the data source to a first dataset stored on the first data node, based on the first authorization rule that authorizes a requesting user to access the first data node; and

load a second data from the data source to a second dataset stored on the second data node based on the second authorization rule that authorizes a requesting user to access the second data node, wherein the first dataset and the second dataset are based on a shared data model;

establishing a plurality of ETL pipelines, each ETL pipeline corresponding to one of the plurality of data nodes, wherein data filtered through each of the ETL pipelines is loaded into a respective dataset stored on a one of the plurality of data nodes corresponding to the ETL pipeline;

receiving a request from a device for data from one of the first data node and the second data node, the request including one of the first user identifier and the second user identifier, the data being requested specified as a query, and an identifier of the one of the first data node and the second data node storing the requested data;

determining whether the requesting device is associated with one of the first user identifier and the second user identifier that is authorized to access the one of the first data node and the second data node storing the requested data;

when the requesting device is associated with one of the first user identifier and the second user identifier that is authorized to access the one of the first data node and the second data node storing the requested data, executing the query on at least the one of the first data node and the second data node storing the requested data; and

transmitting results of the query to the requesting device.

10 . A system for partitioning data based on authorization rules and answering a query, comprising:

a processing circuitry; and

a memory, the memory containing instructions that, when executed by the processing circuitry, configure the system to:

initiate a plurality of data nodes that includes at least a first data node and a second data node, wherein the first data node corresponds to a first authorization rule, and the second data node corresponds to a second authorization rule, wherein the first authorization rule authorizes a first user identifier to access the first data node, and the second authorization rule authorizes a second user identifier to access the second data node; and

prior to receiving a user request for data, configure an extract, transform, and load (ETL) application service to:

extract data from a data source;

load a first data from the data source to a first dataset stored on the first data node, based on the first authorization rule that authorizes a requesting user to access the first data node; and

load a second data from the data source to a second dataset stored on the second data node based on the second authorization rule that authorizes a requesting user to access the second data node, wherein the first dataset and the second dataset are based on a shared data model;

establish a plurality of ETL pipelines, each ETL pipeline corresponding to one of the plurality of data nodes, wherein data filtered through each of the ETL pipelines is loaded into a respective dataset stored on a one of the plurality of data nodes corresponding to the ETL pipeline;

receive a request from a device for data from one of the first data node and the second data node, the request including one of the first user identifier and the second user identifier, the data being requested specified as a query, and an identifier of the one of the first data node and the second data node storing the requested data;

determine whether the requesting device is associated with one of the first user identifier and the second user identifier that is authorized to access the one of the first data node and the second data node storing the requested data;

when the requesting device is associated with one of the first user identifier and the second user identifier that is authorized to access the one of the first data node and the second data node storing the requested data, executing the query on at least the one of the first data node and the second data node storing the requested data; and

transmitting results of the query to the requesting device.

11 . The system of claim 10 , wherein the data source is a first data source of a plurality of data sources, and wherein to establish the plurality of ETL pipelines the system is further configured to:

extract data from the plurality of data sources; and

load the extracted data to the dataset stored on the first data node based on the first authorization rule.

12 . The system of claim 10 , wherein each of the first authorization rule and the second authorization rule further authorizes a respective user identifier to access at least one location of data.

13 . The system of claim 12 , wherein each of the first authorization rule and the second authorization rule each defines a respective key, wherein the at least one location of data authorized by each of the first authorization rule and the second authorization rule stores the respective key defined in the corresponding authorization rule.

14 . The system of claim 10 , wherein the system is further configured to:

generate a plurality of user groups, wherein each of the plurality of user groups includes at least one user identifier corresponding to one of the authorization rules.

15 . The system of claim 10 , wherein a group of data nodes each correspond to the first authorization rule.

16 . The system of claim 10 , wherein the system is further configured to:

execute the query on both of the first data node and the second data when the received one of the first user identifier and the second user identifier is authorized to access both of the first data node and the second data.

17 . The system of claim 10 , wherein to initiate a first data node the system is further configured to cause an orchestrator to initiate a new machine in a cloud computing environment.