Hybrid serverless computing and durable computing search service
A hybrid serverless computing and durable computing search system provides low-latency search by maintaining domain specific search indices in memory or storage, while also allowing for low-cost service by scaling to zero when not in use. An initial search query is processed by an ephemeral computing instance loaded with code for executing search and which has downloaded relevant search indexes for a given domain. Asynchronously, a longer-lived container is launched and loaded with the relevant search indices. Subsequent search queries for the given domain are handled using the container without the delay associated with downloading the relevant search indices. After a period of inactivity both the container and ephemeral computing instances are scaled to zero.
1 . A system, comprising:
storage devices configured to store documents;
a database storing indexes that index document characteristics of the documents stored in the storage devices; and
one or more computing devices configured to implement on-demand searching capacity for searching documents stored in the storage devices, wherein to implement the on-demand searching capacity, the one or more computing devices are configured to:
in response to receiving an initial query for a given domain,
instantiate an ephemeral computing instance for searching for documents having characteristics within the given domain;
add index files to a memory of the ephemeral computing instance that have been retrieved from the database;
generate a query response for the initial query using the index files that have been added to the memory of the ephemeral computing instance;
asynchronously, cause a container-based computing environment to be launched for processing additional queries for the given domain; and
load the container-based computing environment with index files for the given domain; and
in response to receiving a subsequent query for the given domain within a threshold amount of time from when the initial query was received,
generate a query response using the container-based computing environment to which the index files for the given domain have been loaded.
2 . The system of claim 1 , wherein the one or more computing devices are further configured to:
release the ephemeral computing instance subsequent to the threshold amount of time passing since generating the query response for the initial query; and
shutdown the container-based computing environment in response to not receiving a subsequent query within the threshold amount of time.
3 . The system of claim 1 , wherein launching the container-based computing environment bypasses a container replication process used when launching other container-based computing environments.
4 . The system of claim 1 , wherein:
in response to receiving the initial query for the given domain, the one or more computing devices are further configured to:
launch a pod of ephemeral computing instances for use in performing subsequent queries using the container-based computing environment.
5 . The system of claim 4 , wherein the pod comprises:
an ephemeral computing instance for asynchronously launching the container-based computing environment; and
an ephemeral computing instance for generating the query response using the container-based computing environment.
6 . The system of claim 5 , wherein the ephemeral computing instance for asynchronously launching the container-based computing environment, is configured to:
in response to successfully launching the container-based computing environment, update one or more entries of the database for the given domain to include a network address for the container-based computing environment.
7 . The system of claim 6 , wherein the ephemeral computing instance for searching for documents having characteristics within the given domain is configured to:
retrieve the network address for the container-based computing environment from the database in response to the receiving the subsequent query for the given domain; and
cause the ephemeral computing instance for generating the query response using the container-based computing environment to generate a query response using the container-based computing environment located at the retrieved network address.
8 . The system of claim 1 , wherein the one or more computing devices are further configured to, in response to receiving the initial query for the given domain:
asynchronously, cause a second container-based computing environment to be launched for processing additional queries for the given domain;
load the second container-based computing environment with updated index files for the given domain while the container-based computing environment is being used to generate query responses for the given domain; and
alternate between using the container-based computing environment and the second container-based computing environment for generating query responses for the given domain, wherein a given one is used for generating query responses while the other given one is loaded with most recently updated index files.
9 . The system of claim 8 , wherein the one or more computing devices are further configured to:
replace a given one of the container-based computing environments not currently being used for generating query responses with a larger or smaller container-based computing environment while continuing to generate query responses using the other given one of the container-based computing environments.
10 . A method, comprising:
receiving an initial query for a given domain at a hybrid serverless computing and durable computing search system;
generating, in response to receiving the initial query, a query response using indexes for the given domain added to a memory of a serverless computing instance that provides on-demand serverless computing capacity;
providing the generated response for the initial query;
receiving a subsequent query for the given domain at the hybrid serverless computing and durable computing search system;
generating, in response to receiving the subsequent query, a query response using indexes for the given domain loaded into a container-based computing environment, wherein the container-based computing environment was launched responsive to receiving the initial query for the given domain and loaded with the indexes for the given domain; and
providing the generated response for the subsequent query.
11 . The method of claim 10 , wherein the indexes for the given domain comprise:
one or more inverted bitmap indexes for searching for documents based on document characteristics.
12 . The method of claim 11 , wherein at least one of the one or more inverted bitmap indexes is configured to be used to perform an approximate nearest neighbor vector search, and wherein vectors are used to represent characteristics of documents available to be searched.
13 . The method of claim 11 , wherein at least one of the one or more inverted bitmap indexes is configured to be used in a filter operation to filter documents based on characteristics, wherein filter results are used to generate the response to the initial query or the subsequent query.
14 . The method of claim 10 , further comprising:
scaling the serverless computing instance and the container-based computing environment to zero instances in response to not receiving a query for the given domain within a threshold amount of time from when the initial query was received or in response to not receiving another query for the given domain within the threshold amount of time since a last query was received.
15 . The method of claim 10 , further comprising:
scaling up or scaling down a size of the container-based computing environment in response to a change in volume of queries received or in response to a change in a size of the indexes for the given domain.
16 . The method of claim 10 , further comprising:
in response to receiving the initial query for the given domain,
launching a pod of serverless computing instances comprising:
a first additional serverless computing instance for asynchronously launching the container-based computing environment; and
a second additional serverless computing instance for generating the query response using the container-based computing environment;
asynchronously launching, by the first additional serverless computing instance, the container-based computing environment for processing additional queries for the given domain; and
loading, by the first additional serverless computing instance, the container-based computing environment with index files for the given domain.
17 . The method of claim 16 , further comprising:
in response to receiving the subsequent query for the given domain,
retrieving, performed by the serverless computing instance used for generating the query response, a network address for the container-based computing environment;
connecting the second additional serverless computing instance with the container-based computing environment located at the retrieved network address; and
generating a query response, performed by the second additional serverless computing instance, using the container-based computing environment.
18 . One or more non-transitory, computer-readable storage media storing program instructions that, when executed on or across one or more processors, cause the one or more processors to:
generate, in response to receiving an initial query for a given domain at a hybrid serverless computing and durable computing search system, a query response using indexes for the given domain added to a memory of a serverless computing instance that provides on-demand serverless computing capacity;
provide the generated response for the initial query;
generate, in response to receiving a subsequent query for the given domain at the hybrid serverless computing and durable computing search system, a query response using indexes for the given domain loaded into a container-based computing environment, wherein the container-based computing environment was launched responsive to receiving the initial query for the given domain and loaded with the indexes for the given domain; and
provide the generated response for the subsequent query.
19 . The one or more non-transitory, computer-readable storage media of claim 18 , wherein the program instructions, when executed on or across the one or more processors, further cause the one or more processors to:
in response to receiving the initial query for the given domain,
launch a pod of serverless computing instances comprising:
a first additional serverless computing instance for asynchronously launching the container-based computing environment; and
a second additional serverless computing instance for generating the query response using the container-based computing environment.
20 . The one or more non-transitory, computer-readable storage media of claim 18 , wherein the program instructions, when executed on or across the one or more processors, further cause the one or more processors to:
in response to receiving the subsequent query for the given domain,
retrieve a network address for the container-based computing environment; and
connecting the second additional serverless computing instance with the container-based computing environment located at the retrieved network address.