Systems and methods for federated query abstraction
A server system is communicatively connected to a plurality of computing devices and one or more databases. The server system receives one or more queries from a computing device. The one or more queries specify a data source. The server system determines a level of security applicable to a user of the computing device. The server system translates the one or more queries into one or more logical queries according to semantics of the data source, and transmits the logical queries to a query pipeline of the server system. The server system executes the one or more queries against a first database of the one or more databases to retrieve query results from the data source. The server system applies the determined level of security to the query results to obtain one or more data sets, and returns the one or more data sets to the computing device.
1 . A method of querying data, performed at a server system that includes one or more processors and memory, the server system communicatively connected to a plurality of computing devices and one or more databases, the method comprising:
receiving one or more queries from a first computing device of the plurality of computing devices, the one or more queries having a serialized data format and specifying one or more data fields of a data source;
in accordance with receiving the one or more queries:
deserializing the one or more queries into one or more deserialized queries;
combining the one or more deserialized queries with one or more user functions;
determining a level of security applicable to a user of the first computing device according to the one or more user functions;
translating the one or more deserialized queries with the one or more user functions into one or more logical queries according to semantics of the data source;
transmitting the one or more logical queries to a query pipeline of the server system;
executing the one or more queries against a first database of the one or more databases to retrieve query results comprising a plurality of data rows, from the data source;
applying the determined level of security to the query results, including determining a subset of data rows, less than all of the plurality of data rows, according to the determined level of security applicable to the user, to obtain one or more data sets; and
returning the one or more data sets to the first computing device.
2 . The method of claim 1 , further comprising:
receiving information regarding a local state of the first computing device,
wherein translating the one or more deserialized queries with the one or more user functions into the one or more logical queries includes encoding the information of the local state of the first computing device in the one or more logical queries.
3 . The method of claim 2 , wherein the information of the local state of the first computing device includes information of a calculation or a filter that exists locally on the first computing device.
4 . The method of claim 1 , wherein the first computing device is a client device executing a data visualization application.
5 . The method of claim 4 , wherein the first computing device is configured to render data from the one or more data sets as a data visualization and display the data visualization via a display of the first computing device.
6 . The method of claim 1 , wherein:
the plurality of computing devices includes a second computing device executing a data service, the data service being communicatively connected with a client device executing an application with a programmatic interface; and
the method further comprises:
receiving, via the second computing device, a query from the programmatic interface of the client device via an API call, wherein the second computing device is configured to generate a query specification from the received query.
7 . The method of claim 1 , wherein the data source is a published data source.
8 . The method of claim 1 , further comprising:
determining respective availabilities of one or more services of the server system.
9 . The method of claim 1 , further comprising:
obtaining metadata information corresponding to the data source.
10 . A computer system that is communicatively connected to a plurality of computing devices and one or more databases, the computer system comprising:
one or more processors; and
memory coupled to the one or more processors, the memory storing one or more programs configured for execution by the one or more processors, the one or more programs including instructions for:
receiving one or more queries from a first computing device of the plurality of computing devices, the one or more queries having a serialized data format and specifying one or more data fields of a data source;
in accordance with receiving the one or more queries:
deserializing the one or more queries into one or more deserialized queries;
combining the one or more deserialized queries with one or more user functions;
determining a level of security applicable to a user of the first computing device according to the one or more user functions;
translating the one or more deserialized queries with the one or more user functions into one or more logical queries according to semantics of the data source;
transmitting the one or more logical queries to a query pipeline of the computer system;
executing the one or more queries against a first database of the one or more databases to retrieve query results comprising a plurality of data rows, from the data source;
applying the determined level of security to the query results, including determining a subset of data rows, less than all of the plurality of data rows, according to the determined level of security applicable to the user, to obtain one or more data sets; and
returning the one or more data sets to the first computing device.
11 . The computer system of claim 10 , the one or more programs further including instructions for:
receiving information regarding a local state of the first computing device,
wherein the instructions for translating the one or more deserialized queries with the one or more user functions into the one or more logical queries include instructions for encoding the information of the local state of the first computing device in the one or more logical queries.
12 . The computer system of claim 11 , wherein the information of the local state of the first computing device includes information of a calculation or a filter that exists locally on the first computing device.
13 . The computer system of claim 10 , wherein;
the plurality of computing device includes a second computing device executing a data service, the data service being communicatively connected with a client device executing an application with a programmatic interface; and
the one or more programs further include instructions for:
receiving, via the second computing device, a query from the programmatic interface of the client device via an API call, wherein the second computing device is configured to generate a query specification from the received query.
14 . A non-transitory computer-readable storage medium storing one or more programs configured for execution by one or more processors of a server system that is communicatively connected to a plurality of computing devices and one or more databases, the one or more programs comprising instructions for:
receiving one or more queries from a first computing device of the plurality of computing devices, the one or more queries having a serialized data format and specifying one or more data fields of a data source;
in accordance with receiving the one or more queries:
deserializing the one or more queries into one or more deserialized queries;
combining the one or more deserialized queries with one or more user functions;
determining a level of security applicable to a user of the first computing device according to the one or more user functions;
translating the one or more deserialized queries with the one or more user functions into one or more logical queries according to semantics of the data source;
transmitting the one or more logical queries to a query pipeline of the server system;
executing the one or more queries against a first database of the one or more databases to retrieve query results comprising a plurality of data rows, from the data source;
applying the determined level of security to the query results, including determining a subset of data rows, less than all of the plurality of data rows, according to the determined level of security applicable to the user, to obtain one or more data sets; and
returning the one or more data sets to the first computing device.
15 . The non-transitory computer-readable storage medium of claim 14 , the one or more programs comprising instructions for:
determining respective availabilities of one or more services of the server system.
16 . The non-transitory computer-readable storage medium of claim 14 , the one or more programs comprising instructions for:
obtaining metadata information corresponding to the data source.
17 . The method of claim 1 , wherein returning the one or more data sets to the first computing device includes causing the one or more data sets to be rendered as a data visualization.
18 . The method of claim 1 , wherein returning the one or more data sets to the first computing device includes returning the one or more data sets as one or more result tables.
19 . The computer system of claim 13 , wherein:
the one or more queries from the first computing device comprises a first session;
the query from the programmatic interface of the client device comprises a second session; and
the one or more programs include instructions for simultaneously handling the first session and the second session.
20 . The non-transitory computer-readable storage medium of claim 14 , wherein:
the plurality of computing device includes a second computing device executing a data service, the data service being communicatively connected with a client device executing an application with a programmatic interface; and
the one or more programs further include instructions for:
receiving, via the second computing device, a query from the programmatic interface of the client device via an API call, wherein the second computing device is configured to generate a query specification from the received query.