Data asset sharing between accounts at a data processing service using cloud tokens
A data processing service receives indication that a recipient will request access to data assets of a provider and provides a request for credentials from a recipient governance module. The recipient governance module stores a recipient metastore including an object for a provider metastore. In response to determining that the assets are associated with the provider metastore, the service provides a request for credentials to a provider governance module. The provider governance module stores the provider metastore describing data assets of the provider and permissions for accessing data assets. The provider metastore includes a recipient object attached to the data assets with an identifier for the recipient metastore. In response to verifying that the recipient was provided access to the data assets, the service provides a token to the recipient governance module. The service then provides the token to a computing resource to provide access to the data assets.
1 . A method, comprising:
receiving an indication that a recipient user of a data processing service will request access to one or more shared data assets of a provider user of the data processing service by a computing resource of a cloud infrastructure;
providing, to a first data governance module for the recipient user, a request to generate temporary credentials, the first data governance module storing a recipient metastore, wherein the recipient metastore includes a provider object for the provider user indicating an identifier for a provider metastore associated with the provider user;
responsive to determining that the one or more shared assets are associated with the provider metastore, providing, to a second data governance module for the provider user, a request to generate the temporary credentials, the second data governance module storing a provider metastore, the provider metastore describing data assets of the provider user, wherein the provider metastore includes a recipient object attached to the one or more shared data assets of the provider user indicating an identifier for the recipient metastore;
responsive to verifying that the user has access to the one or more shared data assets, providing, by the second data governance module, an access token to the first data governance module; and
providing, by the first data governance module, the access token to the computing resource such that the computing resource accesses the one or more shared data assets using the access token.
2 . The method of claim 1 , further comprising:
receiving, from the provider user, a share request to share the one or more shared data assets with the recipient user; and
responsive to receiving the share request, creating the recipient object in the provider metastore.
3 . The method of claim 2 , wherein the indication that the recipient user will request access to the one or more shared data assets of the provider user is received by the data processing service in response to receiving the share request.
4 . The method of claim 1 , wherein the access token is a short-lived temporary token.
5 . The method of claim 1 , wherein the second data governance module for the provider:
identifies a storage location associated with the one or more shared data assets; and
requests the access token from a corresponding cloud identity and access management (IAM) service of a cloud provider based on the storage location.
6 . The method of claim 1 , wherein the access token provides the recipient user access to the one or more data assets stored in cloud object storage.
7 . The method of claim 1 , wherein the recipient user is a first entity with a first account at the data processing service and the provider user is a second entity with a second account at the data processing service.
8 . A non-transitory computer readable storage medium storing instructions that, when executed by one or more computer processors, causes the one or more computer processors to:
receive an indication that a recipient user of a data processing service will request access to one or more shared data assets of a provider user of the data processing service by a computing resource of a cloud infrastructure;
provide, to a first data governance module for the recipient user, a request to generate temporary credentials, the first data governance module storing a recipient metastore, wherein the recipient metastore includes a provider object for the provider user indicating an identifier for a provider metastore associated with the provider user;
responsive to determining that the one or more shared assets are associated with the provider metastore, provide, to a second data governance module for the provider user, a request to generate the temporary credentials, the second data governance module storing a provider metastore, the provider metastore describing data assets of the provider user, wherein the provider metastore includes a recipient object attached to the one or more shared data assets of the provider user indicating an identifier for the recipient metastore;
responsive to verifying that the user has access to the one or more shared data assets, provide, by the second data governance module, an access token to the first data governance module; and
provide, by the first data governance module, the access token to the computing resource such that the computing resource accesses the one or more shared data assets using the access token.
9 . The non-transitory computer readable storage medium of claim 8 , wherein the instructions that, when executed by the one or more computer processors, further causes the one or more computer processors to:
receive, from the provider user, a share request to share the one or more shared data assets with the recipient user; and
responsive to receiving the share request, create the recipient object in the provider metastore.
10 . The non-transitory computer readable storage medium of claim 9 , wherein the indication that the recipient user will request access to the one or more shared data assets of the provider user is received by the data processing service in response to receiving the share request.
11 . The non-transitory computer readable storage medium of claim 8 , wherein the access token is a short-lived temporary token.
12 . The non-transitory computer readable storage medium of claim 8 , wherein the second data governance module for the provider:
identifies a storage location associated with the one or more shared data assets; and
requests the access token from a corresponding cloud identity and access management (IAM) service of a cloud provider based on the storage location.
13 . The non-transitory computer readable storage medium of claim 8 , wherein the access token provides the recipient user access to the one or more data assets stored in cloud object storage.
14 . The non-transitory computer readable storage medium of claim 8 , wherein the recipient user is a first entity with a first account at the data processing service and the provider user is a second entity with a second account at the data processing service.
15 . A computer system comprising:
a computer processor; and
a non-transitory computer readable storage medium storing instructions that, when executed by one or more computer processors, causes the one or more computer processors to:
receive an indication that a recipient user of a data processing service will request access to one or more shared data assets of a provider user of the data processing service by a computing resource of a cloud infrastructure;
provide, to a first data governance module for the recipient user, a request to generate temporary credentials, the first data governance module storing a recipient metastore, wherein the recipient metastore includes a provider object for the provider user indicating an identifier for a provider metastore associated with the provider user;
responsive to determining that the one or more shared assets are associated with the provider metastore, provide, to a second data governance module for the provider user, a request to generate the temporary credentials, the second data governance module storing a provider metastore, the provider metastore describing data assets of the provider user, wherein the provider metastore includes a recipient object attached to the one or more shared data assets of the provider user indicating an identifier for the recipient metastore;
responsive to verifying that the user has access to the one or more shared data assets, provide, by the second data governance module, an access token to the first data governance module; and
provide, by the first data governance module, the access token to the computing resource such that the computing resource accesses the one or more shared data assets using the access token.
16 . The computing system of claim 15 , wherein the instructions that, when executed by the one or more computer processors, further causes the one or more computer processors to:
receive, from the provider user, a share request to share the one or more shared data assets with the recipient user; and
responsive to receiving the share request, create the recipient object in the provider metastore.
17 . The computing system of claim 16 , wherein the indication that the recipient user will request access to the one or more shared data assets of the provider user is received by the data processing service in response to receiving the share request.
18 . The computing system of claim 15 , wherein the access token is a short-lived temporary token.
19 . The computing system of claim 15 , wherein the second data governance module for the provider:
identifies a storage location associated with the one or more shared data assets; and
requests the access token from a corresponding cloud identity and access management (IAM) service of a cloud provider based on the storage location.
20 . The computing system of claim 15 , wherein the access token provides the recipient user access to the one or more data assets stored in cloud object storage.