Platform management of integrated access of public and privately-accessible datasets utilizing federated query generation and query schema rewriting optimization
Various techniques are described for platform management of integrated access of public and privately-accessible datasets utilizing federated query generation and query schema rewriting optimization, including receiving at a dataset access platform a query formatted according to a first data schema, generating a copy of the query, saving the query and the copy to a datastore, parsing the copy of the query in the first schema using an inference engine, determining whether the query comprises data associated with an access control condition associated with accessing the dataset, the access control condition being configured to indicate whether the query is permitted to access the dataset, and rewriting, using a proxy server, the copy of the query in a second schema by converting the copy of the query into a triple associated with the query and another triple associated with the access control condition.
1. A method, comprising:
causing to deploy via a network data representing one or more portions of an application distributed among computing cloud-based resources, the one or more portions of the application configured to generate one or more federated queries; and
identifying via the network a request to perform a query received in a first data format into at least one of the one or more portions of the application to access a dataset, the one or more portions of the application configured to perform data operations including:
generating a copy of the query to form data representing a query copy in the first data format;
storing either the query or the query copy, or both, to one or more data stores of the computing cloud-based resources;
inferring an attribute associated with the query to form an inferred attribute;
parsing either the query or the query copy, or both, to identify the inferred attribute associated with the query or the query copy;
rewriting the query copy to convert into a second data format to form a rewritten query in the second data format; and
modifying a graph to form one or more data links between the dataset and another dataset based on data representing the rewritten query.
2. The method of claim 1 , further comprising:
directing the query to one or more endpoints associated with a dataset access platform to retrieve query results associated with a target database configured to store the dataset as graph-based data.
3. The method of claim 1 , further comprising:
rewriting of the query copy to generate a federated query that is configured to access one or more endpoints.
4. The method of claim 1 , wherein data representing the inferred attribute is a type of attribute including access control data.
5. The method of claim 1 , wherein data representing the query includes data representing a security-related attribute as access control data.
6. The method of claim 1 , wherein parsing either the query or the query copy, or both, comprises:
determining the query comprises other data including authentication data to access the dataset.
7. The method of claim 1 , wherein the first data format is associated with either a structure or an unstructured data schema, or both.
8. The method of claim 1 , wherein the first data format is associated with a relational data schema.
9. The method of claim 1 , wherein the first data format is associated with a schema compatible with a structured query language (“SQL”) or equivalent thereto.
10. The method of claim 1 , wherein the second data format is associated with a triples-based format.
11. The method of claim 1 , wherein the second data format is associated with a resource description framework (“RDF”) format.
12. The method of claim 1 , wherein the query is a master query.
13. The method of claim 1 , further comprising:
receiving the query into at least a portion of a dataset access platform implemented in the one or more portions of the application.
14. The method of claim 1 , further comprising:
rewriting the query copy at least a portion of a proxy server implemented in the one or more portions of the application.
15. The method of claim 1 , further comprising:
retrieving query results from a target database configured to store the dataset as graph-based data.
16. A system comprising:
a memory including executable instructions; and
a processor configured to execute the instructions to:
cause to deploy via a network data representing one or more portions of an application distributed among computing cloud-based resources, the one or more portions of the application configured to generate one or more federated queries; and
identify via the network a request to perform a query received in a first data format into at least one of the one or more portions of the application to access a dataset, the one or more portions of the application configured to perform data operations including:
generating a copy of the query to form data representing a query copy in the first data format;
storing either the query or the query copy, or both, to one or more data stores of the computing cloud-based resources;
inferring an attribute associated with the query to form an inferred attribute;
parsing either the query or the query copy, or both, to identify the inferred attribute associated with the query or the query copy;
rewriting the query copy to convert into a second data format to form a rewritten query in the second data format; and
modifying a graph to form one or more data links between the dataset and another dataset based on data representing the rewritten query.
17. The system of claim 16 , wherein the processor is further configured to:
direct the query to one or more endpoints associated with a dataset access platform to retrieve query results associated with a target database configured to store the dataset as graph-based data.
18. The system of claim 16 , wherein the processor is further configured to:
rewrite of the query copy to generate a federated query that is configured to access one or more endpoints.
19. The system of claim 16 , wherein data representing the inferred attribute is a type of attribute including access control data.
20. The system of claim 16 , wherein the processor configured to parse either the query or the query copy, or both, is configured further to:
determine the query comprises other data including authentication data to access the dataset.