Access control of managed clusters performing data processing on cloud platforms
A system manages access control for database queries in accordance with access control policies. The techniques may be used for access control of managed clusters in cloud platforms used for data processing, for example, for MapReduce operations. According to an embodiment, the access control policies are fine grained access control policies that allow a user to access a subset of datasets including data of a dataset. The system receives and compiles a database query to generate a query plan for processing the database query. The query plan includes: one or more data access operators, one or more data processing operators, and one or more access control filters. The system executes the query plan using an executor process and a helper process. The helper process executes a data access operator and a corresponding data access filter and the executor process executes one or more data processing operators.
1 . A method of managing access control for database queries, the method comprising:
receiving an access control policy specification of an access control policy comprising a condition specified as an expression based on one or more attributes describing data of a first dataset of a database;
receiving a database query for processing data obtained from one or more datasets comprising the first dataset;
compiling the database query to generate a query plan in accordance with the access control policy, the query plan comprising:
one or more data access operators, each data access operator configured to access a dataset of the database,
one or more data processing operators, each data processing operator configured to process at least a subset of data accessed by a data access operator, and
an access control filter that determines a subset of data accessed by a data access operator from a dataset and provides as input to a data processing operator, the subset of data determined according to the access control policy, the access control filter configured to determine a subset of data of the first dataset according to the condition of the first dataset;
executing the query plan using at least an executor process and a helper process, wherein the helper process executes a data access operator and a corresponding data access filter and the executor process executes one or more data processing operators, wherein the executor process and the helper process are prevented from accessing the one or more datasets before executing the query plan; and
providing temporary credentials to the helper process for accessing datasets processed by the database query for executing the query plan, wherein the temporary credentials provide access to more data of a dataset than is permitted by the access control policy.
2 . The method of claim 1 , further comprising:
generating the access control filter according to the access control policy specification.
3 . The method of claim 2 , wherein the access control filter processes data of a first data access operator processing the first dataset.
4 . The method of claim 1 , wherein data accessed by a first data access operator configured to access the first dataset is provided to a data processing operator without an access control filter if the access control policy specification does not include any condition based on attributes of the first dataset.
5 . The method of claim 1 , wherein the helper process provides the subset of data of the dataset obtained by executing the data access operator and the corresponding data access filter to the executor process using an interprocess communication mechanism.
6 . The method of claim 1 , wherein the dataset is stored in a file and the temporary credentials grant access to the helper process to all data of the file.
7 . The method of claim 1 , wherein the access control policy requires at least a first subset of data to be masked before providing to a user, wherein a first access control filter includes instructions to mask the first subset of data.
8 . The method of claim 1 , wherein the access control policy implements one or more of:
row level access to a dataset;
column level access to a dataset; or
access to data with masking.
9 . A non-transitory computer readable storage medium comprising stored program code, the stored program code comprising instructions, the instructions when executed by one or more computer processors, cause the one or more computer processors to:
receive an access control policy specification of an access control policy comprising a condition specified as an expression based on one or more attributes describing data of a first dataset of a database;
receive a database query for processing data obtained from one or more datasets comprising the first dataset;
compile the database query to generate a query plan in accordance with the access control policy, the query plan comprising:
one or more data access operators, each data access operator configured to access a dataset of the database,
one or more data processing operators, each data processing operator configured to process at least a subset of data accessed by a data access operator, and
an access control filter that determines a subset of data accessed by a data access operator from a dataset and provides as input to a data processing operator, the subset of data determined according to the access control policy, the access control filter configured to determine a subset of data of the first dataset according to the condition of the first dataset;
execute the query plan using at least an executor process and a helper process, wherein the helper process executes a data access operator and a corresponding data access filter and the executor process executes one or more data processing operators, wherein the executor process and the helper process are prevented from accessing the one or more datasets before executing the query plan; and
provide temporary credentials to the helper process for accessing datasets processed by the database query for executing the query plan, wherein the temporary credentials provide access to more data of a dataset than is permitted by the access control policy.
10 . The non-transitory computer readable storage medium of claim 9 , wherein the instructions further cause the one or more computer processors to:
generate the access control filter according to the access control policy specification.
11 . The non-transitory computer readable storage medium of claim 10 , wherein the access control filter processes data of a first data access operator processing the first dataset.
12 . The non-transitory computer readable storage medium of claim 10 , wherein data accessed by a first data access operator configured to access the first dataset is provided to a data processing operator without an access control filter if the access control policy specification does not include any condition based on attributes of the first dataset.
13 . The non-transitory computer readable storage medium of claim 9 , wherein the access control policy implements one or more of:
row level access to a dataset;
column level access to a dataset; or
access to data with masking.
14 . The non-transitory computer readable storage medium of claim 9 , wherein the helper process provides the subset of data of the dataset obtained by executing the data access operator and the corresponding data access filter to the executor process using an interprocess communication mechanism.
15 . The non-transitory computer readable storage medium of claim 9 , wherein the dataset is stored in a file and the temporary credentials grant access to the helper process to all data of the file.
16 . The non-transitory computer readable storage medium of claim 9 , wherein the access control policy requires at least a first subset of data to be masked before providing to a user, wherein a first access control filter includes instructions to mask the first subset of data.
17 . A computer system comprising:
one or more computer processors; and
a non-transitory computer readable storage medium comprising stored program code, the stored program code comprising instructions, the instructions when executed by the one or more computer processors, cause the one or more computer processors to:
receive an access control policy specification of an access control policy comprising a condition specified as an expression based on one or more attributes describing data of a first dataset of a database;
receive a database query for processing data obtained from one or more datasets comprising the first dataset;
compile the database query to generate a query plan in accordance with the access control policy, the query plan comprising:
one or more data access operators, each data access operator configured to access a dataset of the database,
one or more data processing operators, each data processing operator configured to process at least a subset of data accessed by a data access operator, and
an access control filter that determines a subset of data accessed by a data access operator from a dataset and provides as input to a data processing operator, the subset of data determined according to the access control policy, the access control filter configured to determine a subset of data of the first dataset according to the condition of the first dataset;
execute the query plan using at least an executor process and a helper process, wherein the helper process executes a data access operator and a corresponding data access filter and the executor process executes one or more data processing operators, wherein the executor process and the helper process are prevented from accessing the one or more datasets before executing the query plan; and
provide temporary credentials to the helper process for accessing datasets processed by the database query for executing the query plan, wherein the temporary credentials provide access to more data of a dataset than is permitted by the access control policy.
18 . The computer system of claim 17 , wherein the instructions further cause the one or more computer processors to:
generate the access control filter according to the access control policy specification.
19 . The computer system of claim 18 , wherein the access control filter processes data of a first data access operator processing the first dataset.
20 . The computer system of claim 18 , wherein data accessed by a first data access operator configured to access the first dataset is provided to a data processing operator without an access control filter if the access control policy specification does not include any condition based on attributes of the first dataset.