System and algorithms for fast and scalable data access to high dimensionality with sparse data

1 . A system, comprising:

one or more processors configured to:

receive a query;

determine, based at least in part on the query, a plurality of partitions of a hypercube that are expected to comprise information used in generating a response for the query, wherein the plurality of partitions is determined based at least in part on a segmentation of an incoming grid definition defined based at least in part on the query;

obtain, from a data source, a set of results for the plurality of partitions;

determine a response to the query based at least in part on the set of results and a set of dependencies among the plurality of partitions; and

provide the response to the query; and

a memory coupled to the one or more processors and configured to provide the one or more processors with instructions.

2 . The system of claim 1 , wherein determining the response to the query comprises:

determining a plurality of virtual objects for the set of results; and

combining, based at least in part on dependencies among the plurality of partitions, the plurality of virtual objects into the response to the query.

3 . The system of claim 2 , wherein the plurality of virtual objects define a virtual cartesian comprising a plurality of dimensions and for each dimension a set of dimension values.

4 . The system of claim 2 , wherein the plurality of virtual objects define a value map corresponding to an enumeration of cells in the hypercube that are associated with data.

5 . The system of claim 4 , wherein each of the cells in the hypercube enumerated in the value map is identified by a hypercube location vector comprising an array of coordinate values for each of the plurality of dimensions.

6 . The system of claim 4 , wherein the virtual object is converted into a linear cartesian comprising cells for a corresponding partition that have data.

7 . The system of claim 1 , wherein obtaining the set of results for the plurality of partitions comprises:

deconstructing query into a set of calls respectively corresponding to the plurality of partitions; and

obtaining the set of results based on an evaluation of the set of calls.

8 . The system of claim 1 , wherein the response to the query comprises a grid of data that is used to build a report or a sheet.

9 . The system of claim 1 , wherein:

the one or more processors are further configured to communicate a set of requests to a data source storing the hypercube;

the set of requests comprising one or more requests corresponding to the plurality of partitions of the hypercube; and

at least a subset of the set of requests is processed in parallel.

10 . The system of claim 1 , wherein determining the response to the query based at least in part on the set of results and a set of dependencies among the plurality of partitions comprises enforcing a bounding criteria with respect to the response to the query.

11 . The system of claim 10 , wherein the enforcing the bounding criteria with respect to the response to the query includes restricting a number of cells, rows, or columns in the response to a predefined maximum number.

12 . The system of claim 1 , wherein the one or more processors are further configured to:

communicate a set of requests to the data source storing the hypercube, the set of requests comprising one or more requests corresponding to the plurality of partitions of the hypercube, the set of requests corresponding to the set of results.

13 . The system of claim 1 , wherein the plurality of partitions is determined based at least in part on translating the query into a declarative description of locations in the hypercube at which data is to be computed.

14 . The system of claim 1 , wherein the determining the plurality of partitions further comprises:

generating a set of identifiers for coordinates associated with a resulting grid that are to be synthesized from multiple data points in the hypercube.

15 . The system of claim 14 , wherein an identifier of the set of identifiers is generated in response to determining that the query includes an in-tier calculation.

16 . The system of claim 1 , wherein a set of requests for information stored in the plurality of partitions is determined based at least in part on the dependencies and coordinates associated with the segment intersections.

17 . A method, comprising:

receiving, by one or more processors, a query;

determining, based at least in part on the query, a plurality of partitions of a hypercube that are expected to comprise information used in generating a response for the query, wherein the plurality of partitions is determined based at least in part on a segmentation of an incoming grid definition defined based at least in part on the query;

obtaining, from a data source, a set of results for the plurality of partitions;

determining a response to the query based at least in part on the set of results and a set of dependencies among the plurality of partitions; and

providing the response to the query.

18 . A computer program product embodied in a non-transitory computer readable medium and comprising computer instructions for:

receiving, by one or more processors, a query;

determining, based at least in part on the query, a plurality of partitions of a hypercube that are expected to comprise information used in generating the response for the query, wherein the plurality of partitions is determined based at least in part on a segmentation of an incoming grid definition defined based at least in part on the query;

obtaining, from a data source, a set of results for the plurality of partitions;

determining a response to the query based at least in part on the set of results and a set of dependencies among the plurality of partitions; and

providing the response to the query.