Referential sampling of polygot datasets

1. One or more non-transitory machine-readable storage mediums storing one or more sequences of instructions for referential sampling of disparate datasets, which when executed by one or more processors, cause:

determining an execution mode and a sampling mode for each entity in a plurality of disparate datasets,

wherein said each entity in the plurality of disparate datasets includes any tables in each of the plurality of disparate datasets,

wherein said execution mode for said each entity in the plurality of disparate datasets may correspond to either a native mode or a non-native mode, wherein said native mode is performed locally with a particular dataset, and wherein said non-native mode is performed remotely to said particular dataset, and

wherein said sampling mode for said each entity in the plurality of disparate datasets may correspond to either a direct mode or a referential mode, wherein the direct mode is used for sampling a particular dataset on its own without reference to another dataset, and wherein said referential mode is used for sampling said particular dataset in relation to other datasets;

creating a directed acyclic graph (DAG) for each entity in the plurality of disparate datasets;

topologically sorting the directed acyclic graph (DAG) to produce a topologically sorted directed acyclic graph (DAG); and

retrieving one or more sampled datasets from the plurality of disparate datasets using the topologically sorted directed acyclic graph (DAG),

wherein the one or more sampled datasets constitute a consistent sample that honors all referential constraints in the plurality of disparate datasets.

2. The one or more non-transitory machine-readable storage mediums of claim 1 , wherein execution of the one or more sequences of instructions further cause:

prior to retrieving said one or more sampled datasets using said topologically sorted directed acyclic graph (DAG), optimizing said topologically sorted directed acyclic graph (DAG) to ensure that, to the extent possible, all dependencies of each node of the topologically sorted directed acyclic graph DAG can be sampled while said each node is in memory.

3. The one or more non-transitory machine-readable storage mediums of claim 1 , wherein retrieving said sampled dataset from the plurality of disparate datasets using the topologically sorted directed acyclic graph (DAG) comprises:

traversing the topologically sorted directed acyclic graph (DAG) to retrieve data from entities in said plurality of disparate datasets in an order based on nodes of the topologically sorted directed acyclic graph (DAG).

4. The one or more non-transitory machine-readable storage mediums of claim 1 , wherein the number of datasets in said one or more sampled datasets retrieved from the plurality of disparate datasets is equal to the number of datasets in said plurality of disparate datasets.

5. An apparatus for referential sampling of disparate datasets, comprising:

one or more processors; and

one or more non-transitory computer-readable storage mediums storing one or more sequences of instructions, which when executed, cause:

determining an execution mode and a sampling mode for each entity in a plurality of disparate datasets,

wherein said each entity in the plurality of disparate datasets includes any tables in each of the plurality of disparate datasets,

wherein said execution mode for said each entity in the plurality of disparate datasets may correspond to either a native mode or a non-native mode, wherein said native mode is performed locally with a particular dataset, and wherein said non-native mode is performed remotely to said particular dataset, and

wherein said sampling mode for said each entity in the plurality of disparate datasets may correspond to either a direct mode or a referential mode, wherein the direct mode is used for sampling a particular dataset on its own without reference to another dataset, and wherein said referential mode is used for sampling said particular dataset in relation to other datasets;

creating a directed acyclic graph (DAG) for each entity in the plurality of disparate datasets;

topologically sorting the directed acyclic graph (DAG) to produce a topologically sorted directed acyclic graph (DAG); and

retrieving one or more sampled datasets from the plurality of disparate datasets using the topologically sorted directed acyclic graph (DAG),

wherein the one or more sampled datasets constitute a consistent sample that honors all referential constraints in the plurality of disparate datasets.

6. The apparatus of claim 5 , wherein execution of the one or more sequences of instructions further cause:

prior to retrieving said one or more sampled datasets using said topologically sorted directed acyclic graph (DAG), optimizing said topologically sorted directed acyclic graph (DAG) to ensure that, to the extent possible, all dependencies of each node of the topologically sorted directed acyclic graph DAG can be sampled while said each node is in memory.

7. The apparatus of claim 5 , wherein retrieving said sampled dataset from the plurality of disparate datasets using the topologically sorted directed acyclic graph (DAG) comprises:

traversing the topologically sorted directed acyclic graph (DAG) to retrieve data from entities in said plurality of disparate datasets in an order based on nodes of the topologically sorted directed acyclic graph (DAG).

8. The apparatus of claim 5 , wherein the number of datasets in said one or more sampled datasets retrieved from the plurality of disparate datasets is equal to the number of datasets in said plurality of disparate datasets.

9. A method for referential sampling of disparate datasets, comprising:

determining an execution mode and a sampling mode for each entity in a plurality of disparate datasets,

wherein said each entity in the plurality of disparate datasets includes any tables in each of the plurality of disparate datasets,

wherein said execution mode for said each entity in the plurality of disparate datasets may correspond to either a native mode or a non-native mode, wherein said native mode is performed locally with a particular dataset, and wherein said non-native mode is performed remotely to said particular dataset, and

wherein said sampling mode for said each entity in the plurality of disparate datasets may correspond to either a direct mode or a referential mode, wherein the direct mode is used for sampling a particular dataset on its own without reference to another dataset, and wherein said referential mode is used for sampling said particular dataset in relation to other datasets;

creating a directed acyclic graph (DAG) for each entity in the plurality of disparate datasets;

topologically sorting the directed acyclic graph (DAG) to produce a topologically sorted directed acyclic graph (DAG); and

retrieving one or more sampled dataset from the plurality of disparate datasets using the topologically sorted directed acyclic graph (DAG),

wherein the one or more sampled datasets constitute a consistent sample that honors all referential constraints in the plurality of disparate datasets.

10. The method of claim 9 , further comprising:

prior to retrieving said one or more sampled datasets using said topologically sorted directed acyclic graph (DAG), optimizing said topologically sorted directed acyclic graph (DAG) to ensure that, to the extent possible, all dependencies of each node of the topologically sorted directed acyclic graph DAG can be sampled while said each node is in memory.

11. The method of claim 9 , wherein retrieving said sampled dataset from the plurality of disparate datasets using the topologically sorted directed acyclic graph (DAG) comprises:

traversing the topologically sorted directed acyclic graph (DAG) to retrieve data from entities in said plurality of disparate datasets in an order based on nodes of the topologically sorted directed acyclic graph (DAG).

12. The method of claim 9 , wherein the number of datasets in said one or more sampled datasets retrieved from the plurality of disparate datasets is equal to the number of datasets in said plurality of disparate datasets.