Cloud-based database for spatial data lifecycle management
Methods, systems, and computer-readable media for a cloud-based database for spatial data lifecycle management. A spatial database receives elements of spatial data from a plurality of clients of the distributed spatial database. An individual element of the spatial data comprises one or more location values or one or more spatial objects. The spatial database stores the plurality of elements of spatial data using a plurality of storage resources. The spatial database receives a query. The spatial database determines one or more elements of spatial data matching the query from the plurality of elements of spatial data that were stored using the plurality of storage resources.
1 . A system, comprising:
a plurality of hardware-based storage resources of a database offered to a plurality of clients as a service of a provider network comprising a plurality of services accessible via the Internet; and,
one or more processors and one or more memories, of the database, storing computer-executable instructions that, when executed, cause the one or more processors to implement:
one or more ingestion routers configured to:
receive, from the plurality of clients of the database, a plurality of elements of vector data within a spatial reference frame; and
store the plurality of elements of vector data to the plurality of hardware-based storage resources; and
one or more query processors configured to:
receive a vector data query indicating one or more spatial relationships;
select one or more elements of vector data matching the one or more spatial relationships indicated in the vector data query from the plurality of elements of vector data stored to the plurality of storage resources;
retrieve data based on the selected matching vector data;
generate one or more query results comprising one or more transformations of the retrieved data; and
transmit over a network, responsive to said vector data query, said one or more query results that include one or more transformations of retrieved data.
2 . The system as recited in claim 1 , wherein the one or more memories store computer-executable instructions that, when executed, cause the one or more processors to:
co-locate, based at least in part on a spatial proximity of a first location value and a second location value and using the plurality of storage resources, a first element of the vector data comprising the first location value and a second element of the vector data comprising the second location value using the plurality of storage resources.
3 . The system as recited in claim 1 , wherein the one or more memories store additional computer-executable instructions that, when executed, cause the one or more processors to:
select one or more additional elements of vector data matching the vector data query from a plurality of additional elements of vector data stored using one or more data sources external to the plurality of storage resources; and
join the one or more elements of vector data matching the vector data query and the one or more additional elements of vector data matching the vector data query.
4 . The system as recited in claim 1 , wherein another individual element of the vector data comprises one or more spatial objects.
5 . A method, comprising:
receiving, by one or more ingestion routers of a database, a plurality of elements of vector data within a spatial reference frame, from a plurality of clients of the database;
storing, by the database, the plurality of elements of vector data to a plurality of hardware-based storage resources of the database, wherein the vector data comprises multi-dimensional data;
receiving, by one or more query processors of the database, a vector data query indicating one or more spatial relationships;
selecting, from the plurality of elements of vector data stored to the plurality of storage resources and by the one or more query processors of the database,
one or more elements of vector data matching the one or more spatial relationships indicated in the vector data query;
retrieving data based on the selected matching vector data;
generating, by the one or more query processors of the database, one or more query results comprising one or more transformations of the retrieved data; and
transmitting, over a network responsive to the vector data query, the one or more query results including one or more transformations of the retrieved data.
6 . The method as recited in claim 5 , wherein a first element of the vector data comprising a first location value and a second element of the vector data comprising a second location value are co-located using the plurality of storage resources based at least in part on a spatial proximity of the first location value and the second location value.
7 . The method as recited in claim 5 , further comprising:
retrieving one or more additional elements of vector data matching the query from a plurality of additional elements of vector data stored using one or more data sources external to the plurality of storage resources; and
joining, by the database, the one or more elements of vector data matching the query and the one or more additional elements of vector data matching the query.
8 . The method as recited in claim 7 , further comprising:
migrating, by the database based at least in part on analysis of the query, the one or more elements of vector data matching the query to the one or more data sources external to the plurality of storage resources; or
migrating, by the database based at least in part on the analysis of the query, the one or more additional elements of vector data matching the query to the plurality of storage resources.
9 . The method as recited in claim 5 , wherein the plurality of storage resources comprise a first storage tier and a second storage tier, wherein the first storage tier and the second storage tier differ in one or more performance characteristics, and wherein the method further comprises:
migrating, by the database, at least a portion of the plurality of elements of vector data from the first storage tier to the second storage tier based at least on part on application of a retention policy, wherein the retention policy is applied based at least in part on temporal values associated with the plurality of elements of vector data.
10 . The method as recited in claim 5 , further comprising:
correcting, by the database, one or more erroneous values in the vector data based at least in part on execution of one or more functions.
11 . The method as recited in claim 5 , further comprising:
adding, by the database, one or more additional values to one or more of the elements of vector data based at least in part on execution of one or more functions.
12 . The method as recited in claim 5 , wherein the database is offered to the plurality of clients as a service of a provider network comprising a plurality of services accessible via the Internet.
13 . The method as recited in claim 5 , wherein the plurality of clients comprise a first client and a second client, wherein the plurality of elements of vector data comprise a first subset provided by the first client and a second subset provided by the second client, wherein the first subset is stored separately from the second subset in the plurality of storage resources, wherein the query is provided by the first client, and wherein the one or more elements of vector data matching the query are selected from the first subset and not from the second subset.
14 . The method as recited in claim 5 , wherein the vector data comprises elements representing one or more points, one or more lines, or one or more polygons.
15 . One or more non-transitory computer-readable storage media storing program instructions that, when executed on or across one or more processors, perform:
receiving, by one or more ingestion routers of a distributed database, a plurality of elements of vector data within a spatial reference frame, the plurality of elements from a plurality of clients of the distributed database, wherein the distributed database is offered to the plurality of clients as a service of a provider network comprising a plurality of services accessible via the Internet;
storing, by the distributed database, the plurality of elements of vector data using a plurality of hardware-based storage resources of the database, wherein the vector data comprises multi-dimensional data;
receiving, by one or more query processors of the distributed database, a vector data query indicating one or more spatial relationships;
selecting, by the one or more query processors of the distributed database, one or more elements of vector data matching the one or more spatial relationships in the vector data query from the plurality of elements of vector data stored using the plurality of storage resources;
retrieving data based on the selected matching vector data;
generating, by the one or more query processors of the distributed database, one or more query results comprising one or more transformations of the retrieved data; and
sending, over a network responsive to the vector data query, the one or more query results including one or more transformations of the retrieved data.
16 . The one or more non-transitory computer-readable storage media as recited in claim 15 , wherein a first element of the vector data comprising a first location value and a second element of the vector data comprising a second location value are co-located using the plurality of storage resources based at least in part on a proximity of the first location value and the second location value.
17 . The one or more non-transitory computer-readable storage media as recited in claim 15 , further comprising additional program instructions that, when executed on or across the one or more processors, perform:
retrieving one or more additional elements of vector data matching the query from a plurality of additional elements of vector data stored using one or more data sources external to the plurality of storage resources; and
joining, by the distributed database, the one or more elements of vector data matching the query and the one or more additional elements of vector data matching the query.
18 . The one or more non-transitory computer-readable storage media as recited in claim 17 , further comprising additional program instructions that, when executed on or across the one or more processors, perform:
migrating, by the distributed database based at least in part on analysis of the query, the one or more elements of vector data matching the query to the one or more data sources external to the plurality of storage resources; or
migrating, by the distributed database based at least in part on the analysis of the query, the one or more additional elements of vector data matching the query to the plurality of storage resources.
19 . The one or more non-transitory computer-readable storage media as recited in claim 15 , wherein the plurality of storage resources comprise a first storage tier and a second storage tier, wherein the first storage tier and the second storage tier differ in one or more performance characteristics, and wherein the one or more non-transitory computer-readable storage media further comprise additional program instructions that, when executed on or across the one or more processors, perform:
migrating, by the distributed database, at least a portion of the plurality of elements of vector data from the first storage tier to the second storage tier based at least on part on application of a retention policy, wherein the retention policy is applied based at least in part on temporal values associated with the plurality of elements of vector data.
20 . The one or more non-transitory computer-readable storage media as recited in claim 15 , wherein the plurality of clients comprise a first client and a second client, wherein the plurality of elements of vector data comprise a first subset provided by the first client and a second subset provided by the second client, wherein the first subset is stored separately from the second subset in the plurality of storage resources, wherein the query is provided by the first client, and wherein the one or more elements of vector data matching the query are selected from the first subset and not from the second subset.