Low-latency database analysis using external data sources
A method and system may be implemented for automatically analyzing data in a database. A method for use in a low-latency database analysis system may include generating a schema. The schema may be based on a portion of an external database. The method may include storing the schema in an in-memory database. The method may include receiving a data-query. The method may include generating a resolved-request. The resolved-request may be based on the data-query and the stored schema. The stored schema may be used for executing the query on the external database. The method may include receiving results data responsive to the data-query from the external database. The method may include outputting the results data for display on a user interface.
1. A method comprising:
generating, in an in-memory database of a database analysis system, an unpopulated database schema corresponding to a populated database schema stored in an external database;
sending, by the database analysis system, to the external database, a data query generated by the database analysis system in accordance with the unpopulated database schema;
obtaining, by the database analysis system, results data output by the external database in response to executing the data query; and
outputting the results data.
2. The method of claim 1 , wherein sending the data query includes:
generating the data query, by the database analysis system, in response to receiving, by the database analysis system, a request for data.
3. The method of claim 2 , wherein generating the data query includes:
determining that an unpopulated table in the unpopulated database schema corresponds to a populated table in the external database.
4. The method of claim 1 , wherein generating the unpopulated database schema includes generating the unpopulated database schema by an external database compute unit of the database analysis system.
5. The method of claim 1 , wherein:
obtaining the results data includes obtaining the results data by a semantic interface of the database analysis system; and
outputting the results data includes:
obtaining the results data by an interface unit of the database analysis system from the semantic interface;
generating, by the interface unit, output data including at least a portion of the results data; and
outputting the output data for presentation to a user.
6. The method of claim 1 , wherein sending the data query to the external database includes:
determining that an override mode indicating external execution is configured.
7. The method of claim 1 , wherein sending the data query to the external database includes:
determining that a previous external execution by the external database is associated with the data query.
8. The method of claim 1 , wherein sending the data query to the external database includes:
determining that cached data for a column indicated by the data query is unavailable from the in-memory database of a database analysis system.
9. A database analysis system comprising:
an in-memory database;
a memory storing instructions for low-latency database analysis using an external data source; and
a processor that executes the instructions to:
generate, in the in-memory database, an unpopulated database schema corresponding to a populated database schema stored in an external database;
send, to the external database, a data query generated by the database analysis system in accordance with the unpopulated database schema;
obtain results data output by the external database in response to execution of the data query; and
output the results data.
10. The database analysis system of claim 9 , wherein to send the data query the processor executes the instructions to:
receive a request for data; and
generate the data query in response to the request for data.
11. The database analysis system of claim 10 , wherein to generate the data query the processor executes the instructions to:
determine that an unpopulated table in the unpopulated database schema corresponds to a populated table in the external database.
12. The database analysis system of claim 9 , wherein to generate the unpopulated database schema the processor executes the instructions to operate an external database compute unit of the database analysis system to generate the unpopulated database schema.
13. The database analysis system of claim 9 , wherein:
to obtain the results data the processor executes the instructions to operate a semantic interface of the database analysis system to obtain the results data by; and
to output the results data the processor executes the instructions to:
operate an interface unit of the database analysis system to:
obtain the results data from the semantic interface;
generate output data including at least a portion of the results data; and
output the output data for presentation to a user.
14. The database analysis system of claim 9 , wherein to send the data query to the external database the processor executes the instructions to:
determine that an override mode indicates external execution is configured.
15. The database analysis system of claim 9 , wherein to send the data query to the external database the processor executes the instructions to:
determine that a previous external execution by the external database is associated with the data query.
16. The database analysis system of claim 9 , wherein to send the data query to the external database the processor executes the instructions to:
determine that cached data for a column indicated by the data query is unavailable from the in-memory database.
17. A non-transitory computer-readable storage medium including executable instructions which when executed by one or more processors cause the one or more processors to perform operations comprising:
generating, in an in-memory database of a database analysis system, an unpopulated database schema corresponding to a populated database schema stored in an external database;
sending, by the database analysis system, to the external database, a data query generated by the database analysis system in accordance with the unpopulated database schema;
obtaining, by the database analysis system, results data output by the external database in response to executing the data query; and
outputting the results data.
18. The non-transitory computer-readable storage medium of claim 17 , wherein sending the data query includes:
generating the data query, by the database analysis system, in response to receiving, by the database analysis system, a request for data.
19. The non-transitory computer-readable storage medium of claim 18 , wherein generating the data query includes:
determining that an unpopulated table in the unpopulated database schema corresponds to a populated table in the external database.
20. The non-transitory computer-readable storage medium of claim 17 , wherein generating the unpopulated database schema includes generating the unpopulated database schema by an external database compute unit of the database analysis system.