Edge-distributed query processing in value chain networks
A method for processing a query for data stored in a distributed database includes receiving, at an edge device, the query for data stored in the distributed database from a query device. The query is a request for data stored at the edge device and for data stored at other edge devices. The method includes executing, by the edge device, the query to find partial query results comprising the data stored at the edge device. The method includes generating, by the edge device, statistical information based on the partial query results. The method includes determining, by the edge device, a statistical confidence associated with the partial results based on the statistical information. The method includes generating, by the edge device, an approximate response to the query based on the statistical information. The method includes transmitting the approximate response to the query device.
1. A method for processing a query for data stored in a distributed database, the method comprising:
receiving, at an edge device, the query for data stored in the distributed database from a query device, wherein the query is a request for data stored at the edge device and for data stored at other edge devices;
executing, by the edge device, the query to find partial query results including the data stored at the edge device;
generating, by the edge device, statistical information based on the partial query results;
generating, by the edge device, a probability distribution model based on the partial query results, wherein the probability distribution model is configured to generate an approximate response to the query;
determining, by the edge device, a statistical confidence of the probability distribution model based on the statistical information; and
in response to the statistical confidence exceeding a determined threshold:
generating, using the probability distribution model, the approximate response to the query, and
transmitting, via the edge device, the approximate response to the query device.
2. The method of claim 1 , wherein the query is an Edge Query Language (EDQL) query.
3. The method of claim 1 , wherein:
the query specifies a shard algorithm, and
the shard algorithm specifies a location of the data stored in the distributed database.
4. The method of claim 1 , further comprising causing, by the edge device, the query, the probability distribution model, and the statistical information to be stored on a dynamic ledger.
5. The method of claim 1 , further comprising:
receiving another second query for data stored in the distributed database; and
generating an approximate response to the second query using the probability distribution model.
6. The method of claim 1 , wherein:
the probability distribution model is a neural network, and
the generating the probability distribution model includes training the neural network.
7. The method of claim 1 , further comprising generating a query plan based on the received query.
8. The method of claim 1 , wherein the approximate response to the query is further based on the partial query results.
9. The method of claim 1 , wherein the edge device is an edge device/aggregator.
10. The method of claim 1 , wherein the statistical information includes outlier data.
11. The method of claim 1 , wherein:
the data stored at the edge device and the data stored at the other edge devices includes sensor data, and
the sensor data is collected from a set of sensors connected to at least one of the edge device or the other edge devices.
12. The method of claim 1 , wherein the distributed database includes a mesh network of edge devices.
13. The method of claim 1 , further comprising:
receiving an instruction, from an aggregator, to reproduce a subset of the data stored at the edge device to another second edge device; and
transmitting the subset of the data to the second edge device.
14. The method of claim 1 , wherein the query is a distributed join query.
15. The method of claim 14 , wherein the executing the query to find the partial query results includes using a reference table stored at the edge device to execute the distributed join query.
16. The method of claim 15 , wherein the reference table is a distributed reference table.
17. The method of claim 14 , wherein the distributed join query is executed without network overhead.
18. The method of claim 4 wherein the dynamic ledger is maintained by the distributed database.
19. The method of claim 4 wherein:
the dynamic ledger is stored in edge storage,
the dynamic ledger is maintained by an aggregator, and
the dynamic ledger includes a blockchain.
20. A system for processing a query for data stored in a distributed database, the system comprising:
a query device including at least one processor;
an edge device communicatively coupled to the query device and including at least one processor; and
a set of other edge devices communicatively coupled to the query device and the edge device, wherein each edge device of the set of other edge devices includes at least one processor,
wherein the at least one processor of the edge device is configured to:
receive the query for data stored in the distributed database from the query device, wherein the query is a request for data stored at the edge device and for data stored at the set of other edge devices,
execute the query to find partial query results including the data stored at the edge device,
generate statistical information based on the partial query results,
generate a probability distribution model based on the partial query results, wherein the probability distribution model is configured to generate an approximate response to the query,
determine a statistical confidence of the probability distribution model based on the statistical information, and
in response to the statistical confidence exceeding a determined threshold:
generate, via the probability distribution model, the approximate response to the query, and
transmit the approximate response to the query device.
21. The system of claim 20 , wherein the at least one processor of the edge device is further configured to store on a dynamic ledger the query, the probability distribution model, and the statistical information.