Access provisioning framework with cell-level security control
Example aspects include techniques for provisioning downstream access to requested data within a data lake with cell-level granularity. These techniques include receiving a request for downstream access to filtered data from a data lake, generating a logical view to the data lake based on the request, the logical view restricted to the filtered data, and generating a temporary storage location for storing retrieved data received from the data lake via the logical view. The techniques also include assigning a compute cluster to the logical view, and accessing, via the logical view, by the compute cluster, the filtered data including storing the filtered data within the temporary storage location.
1. A device comprising:
a memory storing instructions; and
at least one processor coupled with the memory and configured to execute the instructions to:
receive a request for downstream access to filtered data from a data lake;
generate a logical view to the data lake based on the request, the logical view restricted to the filtered data;
generate a temporary storage location for storing retrieved data received from the data lake via the logical view;
assign a compute cluster to the logical view;
generate, based on the request, a user group for a downstream organization, the user group providing read-only access to the temporary storage location; and
access, via the logical view, by the compute cluster, the filtered data and store the filtered data within the temporary storage location.
2. The device of claim 1 , wherein the at least one processor is configured to:
receive, from an application associated with the user group, a request for the filtered data within the temporary storage location; and
transmit the filtered data to the application associated with the user group.
3. The device of claim 1 , wherein to generate the logical view for the data lake based on the request, the at least one processor coupled with the memory and configured to execute the instructions to:
generate the logical view to provide row-level access and column-level access to the data lake.
4. The device of claim 1 , wherein to assign the compute cluster to the logical view, the at least one processor coupled with the memory and configured to execute the instructions to:
limit access and visibility of the compute cluster to the user group; and
limit data access to the logical view to the compute cluster.
5. The device of claim 1 , wherein to assign the compute cluster to the logical view, the at least one processor coupled with the memory and configured to execute the instructions to:
determine a cluster type of the compute cluster based upon the request; and
generate the compute cluster having the cluster type.
6. The device of claim 1 , wherein to assign the compute cluster to the logical view, the at least one processor coupled with the memory and configured to execute the instructions to:
determine one or more entities corresponding to the filtered data;
calculate a complexity score based upon an entity size of each entity of the one or more entities;
determine a cluster type of the compute cluster based upon complexity score; and
generate the compute cluster having the cluster type.
7. The device of claim 1 , wherein the request is a first request, and the at least one processor coupled with the memory and configured to execute the instructions to:
receive a second request that modifies one or more entities identified within the first request; and
resize the compute cluster in response to the second request.
8. The device of claim 1 , wherein the at least one processor coupled with the memory and configured to execute the instructions to:
delete the logical view, user group, the compute cluster and temporary storage location based upon an expiration of the request.
9. A method comprising:
receiving a request for downstream access to filtered data from a data lake;
generating a logical view to the data lake based on the request, the logical view restricted to the filtered data;
generating a temporary storage location for storing retrieved data received from the data lake via the logical view;
assigning a compute cluster to the logical view;
generating, based on the request, a user group for a downstream organization, the user group providing read-only access to the temporary storage location; and
accessing, via the logical view, by the compute cluster, the filtered data and storing the filtered data within the temporary storage location.
10. The method of claim 9 , further comprising:
receiving, from an application associated with the user group, a request for the filtered data within the temporary storage location; and
transmitting the filtered data to the application associated with the user group.
11. The method of claim 9 , wherein generating the logical view for the data lake based on the request, comprises:
generating the logical view to provide row-level access and column-level access to the data lake.
12. The method of claim 9 , wherein assigning the compute cluster to the logical view, comprises:
limiting access and visibility of the compute cluster to the user group; and
limiting data access to the logical view to the compute cluster.
13. The method of claim 9 , wherein assigning the compute cluster to the logical view, comprises:
determining a cluster type of the compute cluster based upon the request; and
generating the compute cluster having the cluster type.
14. The method of claim 9 , wherein assigning the compute cluster to the logical view, comprises:
determining one or more entities corresponding to the filtered data;
calculating a complexity score based upon an entity size of each entity of the one or more entities;
determining a cluster type of the compute cluster based upon complexity score; and
generating the compute cluster having the cluster type.
15. The method of claim 9 , wherein the request is a first request, and further comprising:
receiving a second request that modifies one or more entities identified within the first request; and
resizing the compute cluster in response to second request.
16. The method of claim 15 , further comprising:
deleting the logical view, user group, the compute cluster and temporary storage location based upon an expiration of the request.
17. A non-transitory computer-readable device having instructions thereon that, when executed by at least one computing device, causes the at least one computing device to perform operations comprising:
receiving a request for downstream access to filtered data from a data lake;
generating a logical view to the data lake based on the request, the logical view restricted to the filtered data;
generating a temporary storage location for storing retrieved data received from the data lake via the logical view;
assigning a compute cluster to the logical view;
generating, based on the request, a user group for a downstream organization, the user group providing read-only access to the temporary storage location; and
accessing, via the logical view, by the compute cluster, the filtered data and storing the filtered data within the temporary storage location.
18. The non-transitory computer-readable device of claim 17 , wherein the operations further comprise:
receiving, from an application associated with the user group, a request for the filtered data within the temporary storage location; and
transmitting the filtered data to the application associated with the user group.
19. The non-transitory computer-readable device of claim 17 , wherein generating the logical view for the data lake based on the request, comprises:
generating the logical view to provide row-level access and column-level access to the data lake.
20. The non-transitory computer-readable device of claim 17 , wherein assigning the compute cluster to the logical view, comprises:
determining one or more entities corresponding to the filtered data;
calculating a complexity score based upon an entity size of each entity of the one or more entities;
determining a cluster type of the compute cluster based upon complexity score; and
generating the compute cluster having the cluster type.