Granted Patent B1

US 11,308,106 · App. 15/985,270 · Granted Apr 19, 2022

Inventors: Naveen Muralimanohar (San Jose, CA); Bhaven Avalani (Cupertino, CA); Martin Grund (Lafayette, CA); William Michael McCreedy (Seattle, WA); Ippokratis Pandis (Menlo Park, CA); Michalis Petropoulos (San Francisco, CA)

Assignee: Amazon Technologies, Inc.

G06F16/2471G06F16/24539G06F16/24552G06F16/27

Abstract

Claims (67)

1. A system, comprising:

at least one processor; and

a memory to store program instructions which, if performed by the at least one processor, cause the at least one processor to at least:

responsive to a first query to a database at a first query engine:

cause, by the first query engine, different respective query engines to perform one or more sub-queries to obtain respective results for the one or more sub-queries from different storage locations in a data store for the database;

return, by the first query engine, a final result for the first query that includes the results;

store, by the first query engine, the results of one or more sub-queries in the data store; and

responsive to a second query to the database at the first query engine:

obtain, by the first query engine, the stored results directly from the data store instead of causing the different respective query engines to perform the one or more sub-queries to the different storage locations in the data store to obtain the stored results; and

return, by the first query engine, a final result for the second query that includes the stored results.

2. The system of claim 1 , wherein the program instructions further cause the at least one processor to at least:

further responsive to the second query to the database:

generate, by the first query engine, respective lookup keys for the one or more sub-queries; and

access, by the first query engine, a manifest for the stored results according to the respective lookup keys to determine that valid results are stored for the one or more sub-queries;

identify, by the first query engine, a location for the stored results based on the manifest that indicates valid results are stored for the one or more sub-queries; and

obtain, by the first query engine, the stored results from the identified location to include in the final result for the second query.

3. The system of claim 1 , wherein the program instructions further cause the at least one processor to at least:

before the storage of the results of the one or more sub-queries, apply, by the first query engine, one or more caching criteria to determine to store the results.

4. The system of claim 1 , wherein the first query and the second query are received at a processing cluster of data warehouse service of a provider network that implements the first query engine, wherein the data store is a separate storage service in the provider network, and wherein the different respective query engines are implemented as part of a format independent data processing service in the provider network.

5. A method, comprising:

responsive to a first query to a database received at a first query engine, storing results of one or more sub-queries performed by different respective query engines to obtain the results for the first query from different storage locations in a data store for the database; and

responsive to a second query to the database received at the first query engine,

obtaining, by the first query engine, the stored results directly instead of causing the different respective query engines to perform the one or more sub-queries to obtain the stored results; and

including, by the first query engine, the stored results in a final result for the second query.

6. The method of claim 5 , further comprising:

before storing the results of the one or more sub-queries, applying, by the first query engine, one or more caching criteria to determine to store the results.

7. The method of claim 6 , further comprising identifying, by the first query engine, respective locations to store the results.

8. The method of claim 5 , further comprising:

receiving a result of another sub-query from a query engine that performed the other sub-query with respect to a location in the data store; and

applying one or more caching criteria to the result of the other sub-query to determine to discard the result of the other sub-query.

9. The method of claim 5 , further comprising:

further responsive to the second query at the first query engine:

determining, by the first query engine, that another sub-query identified to perform the second query does not have a valid stored result; and

causing, by the first query engine, a query engine to perform the other sub-query to obtain a result for the other sub-query from a location in the data store.

10. The method of claim 5 , wherein the result for a first one of the one or more sub-queries is stored in a first data store and wherein the result for a second one of the one or more sub-queries is stored in a second data store.

11. The method of claim 5 , further comprising:

further responsive to the second query to the database at the first query engine:

generating, by the first query engine, respective lookup keys for the one or more sub-queries;

accessing, by the first query engine, a manifest for the stored results according to the respective lookup keys to determine that valid results are stored for the one or more sub-queries;

identifying, by the first query engine, a location for the stored results based on the manifest that indicates valid results are stored for the one or more sub-queries; and

obtaining, by the first query engine, the stored results from the identified location to include in the final result for the second query.

12. The method of claim 11 , wherein the manifest for the stored results is stored in a first data store that is different than a second data store that stores the results of the one or more sub-queries.

13. The method of claim 5 , further comprising:

removing one or more data objects storing individual ones of the results for the different storage locations according to one or more retention criteria for storing the results.

14. A non-transitory, computer-readable storage medium, storing program instructions that when executed by one or more computing devices cause the one or more computing devices to implement:

responsive to a first query to a database received at a first query engine:

causing, by the first query engine, different respective query engines to perform one or more sub-queries to obtain respective results for the one or more sub-queries from different storage locations in a data store for the database to include a final result returned for the first query;

storing, by the first query engine, the results of one or more sub-queries; and

responsive to a second query to the database received at the first query engine,

obtaining, by the first query engine, the stored results directly instead of causing the different respective query engines to perform the one or more sub-queries to obtain the stored results; and

including, by the first query engine, the stored results in a final result for the second query.

15. The non-transitory, computer-readable storage medium of claim 14 , wherein the program instructions cause the one or more computing devices to further implement:

further responsive to the second query to the database received at the first query engine:

generating, by the first query engine, respective lookup keys for the one or more sub-queries;

accessing, by the first query engine, a manifest for the stored results according to the respective lookup keys to determine that valid results are stored for the one or more sub-queries;

identifying, by the first query engine, a location for the stored results based on the manifest that indicates valid results are stored for the one or more sub-queries; and

obtaining, by the first query engine, the stored results from the identified location to include in the final result for the second query.

16. The non-transitory, computer-readable storage medium of claim 15 , wherein generating respective lookup keys for the one or more sub-queries is based, at least in part, on respective statements for the one or more sub-queries and on respective version identifiers of data stored in the different locations.

17. The non-transitory, computer-readable storage medium of claim 15 , wherein at least one of the stored results is stored as part of an entry for the sub-query of the one or more sub-queries that generated the at least one stored result in the manifest.

18. The non-transitory, computer-readable storage medium of claim 14 , wherein the program instructions further cause the at least one processor to further implement:

further responsive to the second query received at the first query engine:

determining, by the first query engine, that another sub-query identified to perform the second query does not have a valid stored result; and

causing, by the first query engine, another query engine to perform the other sub-query to obtain a result for the other sub-query from a location in the data store.

19. The non-transitory, computer-readable storage medium of claim 14 , wherein the program instructions further cause the at least one processor to further implement:

before storing the results of the one or more sub-queries, applying one or more caching criteria to determine to store the results.

20. The non-transitory, computer-readable storage medium of claim 14 , wherein the program instructions cause the one or more computing devices to further implement:

causing a data store that stores the results of the one or more sub-queries to the different locations in the data store to retain the results for a threshold period of time after which the results are to be deleted.

Assignments (1)

Cited By (9)

US 12,189,625 US 12,287,816 US 12,295,074 US 12,306,835 US 12,339,851 US 12,360,995 US 12,499,113 US 12,566,763 US 12,750,208