Low-latency database analysis using external data sources
A method and system may be implemented for automatically analyzing data in a database. A method for use in a low-latency database analysis system may include generating a schema. The schema may be based on a portion of an external database. The method may include storing the schema in an in-memory database. The method may include receiving a data-query. The method may include generating a resolved-request. The resolved-request may be based on the data-query and the stored schema. The stored schema may be used for executing the query on the external database. The method may include receiving results data responsive to the data-query from the external database. The method may include outputting the results data for display on a user interface.
1. A low-latency database analysis system comprising:
an in-memory database, wherein the in-memory database is a low-latency database;
a semantic interface configured to interface with the in-memory database and interface with an external database;
an external database compute unit configured to generate a schema in the in-memory database based on a portion of the external database;
the semantic interface configured to receive a resolved-request, wherein the semantic interface is further configured to generate a data-query based on the in-memory database schema for execution on the external database on a condition that a previous data-query associated with the data-query was executed on the external database; and
an interface unit configured to receive results data responsive to the data-query from the external database via the semantic interface and output the results data for display on a user interface.
2. The system of claim 1 , wherein the interface unit is further configured to communicate with the semantic interface.
3. The system of claim 1 , wherein the low-latency database analysis system includes a relational search unit configured to communicate with the interface unit and the semantic interface.
4. The system of claim 3 , wherein the relational search unit is configured to fetch a column data value in response to receiving a data change notification.
5. The system of claim 1 , wherein the schema is a physical schema.
6. The system of claim 1 , wherein the in-memory database is configured to store prioritized data from the external database.
7. A low-latency database analysis system comprising:
an in-memory database, wherein the in-memory database is a low-latency database;
a semantic interface configured to interface with the in-memory database and interface with an external database;
an in-memory database manager configured to communicate with the in-memory database;
an external database compute unit configured to generate a schema in the in-memory database based on a portion of the external database;
an external database manager configured to communicate with the external database compute unit and the external database;
the semantic interface configured to receive a resolved-request, wherein the semantic interface is further configured to generate a data-query based on the in-memory database schema for execution on the external database; and
an interface unit configured to receive results data responsive to the data-query from the external database via the semantic interface and output the results data for display on a user interface.
8. The system of claim 7 , wherein the interface unit is further configured to communicate with the semantic interface.
9. The system of claim 7 , wherein the low-latency database analysis system includes a relational search unit configured to communicate with the interface unit and the semantic interface.
10. The system of claim 9 , wherein the relational search unit is configured to fetch a column data value in response to receiving a data change notification.
11. The system of claim 7 , wherein the in-memory database manager is further configured to send a data change notification to a distributed cluster manager.
12. The system of claim 7 , wherein the in-memory database is configured to store prioritized data from the external database.
13. A method for use in a low-latency database analysis system, the method comprising:
generating a schema based on a portion of an external database;
storing the schema in an in-memory database, wherein the in-memory database is a low-latency database;
receiving a resolved-request;
generating a data-query based on the in-memory database schema for execution on the external database;
receiving results data responsive to the data-query from the external database; and
outputting the results data for display on a user interface.
14. The method of claim 13 further comprising:
determining a data change; and
transmitting a data change notification to a distributed cluster manager.
15. The method of claim 14 further comprising:
tracking the data change.
16. The method of claim 13 further comprising:
storing prioritized data from the external database.
17. The method of claim 16 further comprising:
fetching a column data value in response to receiving a data change notification.