Systems and methods for managing large volumes of data in a digital earth environment
A computer-implemented method for managing large volumes of data comprises dividing data about a number of features into a plurality of data groups, each of the groups having a plurality of features, each of the features having a plurality of properties, and each of the properties having a property value; for each of the groups, determining a number of distribution ranges for the property values for each of the properties; for each of the groups, determining a number of features having property values that are within each of the distribution ranges; and generating a summary associated with each of the groups, the summary comprising the properties of the features in the group and the number of the features that are within each of the distribution ranges for the properties.
1. A computer-implemented method for processing queries comprising:
(a) receiving a query;
(b) receiving at least one summary associated with one or more data groups, each of the groups having a plurality of features, each of the features having a plurality of properties, each of the properties having a property value, and the at least one summary including the properties of the features in the group and the number of the features that are within each of the distribution ranges for the properties;
(c) conducting the query using the at least one summary to obtain a preliminary response to the query; and
(d) generating a margin of error associated with the preliminary response based upon the distribution ranges for the properties in the at least one summary.
2. A computer-implemented method for processing queries, comprising:
(a) receiving a query;
(b) receiving at least one summary associated with one or more data groups, each of the groups having a plurality of features, each of the features having a plurality of properties, each of the properties having a property value, and the at least one summary including the properties of the features in the group and the number of the features that are within each of the distribution ranges for the properties;
(c) conducting the query using the at least one summary to obtain a preliminary response to the query;
(d) generating a margin of error associated with the preliminary response based upon the distribution ranges for the properties in the at least one summary;
(e) receiving at least one additional lower-level summary that is of a different level than a previously received summary;
(f) conducting the query using the at least one lower-level summary to obtain a second preliminary response to the query; and
(g) generating a margin of error associated with the second preliminary response based upon the margin of error associated with the at least one summary.
3. The method of claim 2 , further comprising repeating the steps of claim 2 until a preliminary response within an acceptable margin of error is obtained.
4. The method of claim 2 , further comprising determining which lower-level summary to receive based upon the at least one previously summary.
5. An information system comprising:
(a) at least one server having access to a hierarchical levels of summaries associated with one or more data groups, each of the groups having a plurality of features, each of the features having a plurality of properties, each of the properties having a property value, and the at least one summary including the properties of the features in the group and the number of the features that are within each of the distribution ranges for the properties; and
(b) at least one electronic device in data communication with the server, the at least one electronic device configured to:
(i) receive a query,
(ii) receive at least one summary from the at least one server,
(iii) conduct the query using the at least one summary to obtain a preliminary response to the query, and
(iv) generate a margin of error associated with the preliminary response based upon the margin of error associated with the at least one summary.
6. The system of claim 5 , wherein the data groups are aligned on a common geospatial grid.
7. The system of claim 6 , wherein the summaries can be divided and combined for transmission thereof.
8. The system of claim 5 , wherein the at least one electronic device comprises a plurality of devices in data communication with each other via a peer to peer network.
9. An information system comprising:
(a) at least one server having access to a hierarchical levels of summaries associated with one or more data groups, each of the groups having a plurality of features, each of the features having a plurality of properties, each of the properties having a property value, and the at least one summary including the properties of the features in the group and the number of the features that are within each of the distribution ranges for the properties;
(b) at least one electronic device in data communication with the server, the at least one electronic device configured to:
(i) receive a query,
(ii) receive at least one summary from the at least one server,
(iii) conducting the query using the at least one summary to obtain a preliminary response to the query, and
(iv) generating a margin of error associated with the preliminary response based upon the margin of error associated with the at least one summary,
(v) receiving at least one additional lower-level summary that is of a different level than a previously received summary,
(vi) conducting the query using the at least one lower-level summary to obtain a second preliminary response to the query, and
(vii) generating a margin of error associated with the second preliminary response based upon the margin of error associated with the at least one summary.
10. The system of claim 9 , wherein the data groups are aligned on a common geospatial grid.
11. The system of claim 10 , wherein the summaries can be divided and combined for transmission thereof.
12. The system of claim 9 , wherein the at least one electronic device comprises a plurality of devices in data communication with each other via a peer to peer network.