Query generation for collaborative datasets
Various embodiments relate generally to data science and data analysis, computer software and systems, and wired and wireless network communications to provide an interface between repositories of disparate datasets and computing machine-based entities that seek access to the datasets, and, more specifically, to a computing and data storage platform that facilitates consolidation of one or more datasets, whereby a collaborative data layer and associated logic facilitate, for example, efficient access to, and implementation of, collaborative datasets. In some examples, a method may include receiving data representing a query of a consolidated dataset that may include datasets formatted atomized datasets, analyzing the query to classify portions of the query to form classified query portions, partitioning the query into sub-queries as a function of a classification type for each of the classified query portions, and retrieving data representing a query result from distributed data repositories.
1. A method comprising:
receiving ingested data at a computing system including one or more processors configured to execute instructions to implement one or more executable modules, the ingested data representing a plurality of datasets, each dataset having a different data format as ingested into a dataset ingestion controller module;
formatting a first dataset of the plurality of datasets to form the first dataset in a collaborative data format;
formatting a second dataset of the plurality of datasets to form the second dataset in the collaborative data format;
implementing the one or more executable modules in a collaborative dataset consolidation system configured to form a collaborative dataset based on the collaborative data format;
determining attributes of the collaborative dataset, a first attribute being associated with an account identifier and a second attribute identifying a first version of the collaborative dataset;
interpreting a subset of ingested data of the dataset against one or more data classifications at an inference engine module to derive at least an inferred attribute for the subset of data;
updating the collaborative dataset to link to data associated with the ingested data; and
transmitting activity feed data to disseminate an update to a community of networked users associated with the account identifier, the update including data representing generation of the first version of the collaborative dataset.
2. The method of claim 1 further comprising:
generating a second version of the collaborative dataset responsive to the link to the data associated with the ingested data including the inferred attribute;
deducing a data classification for the subset of ingested data of the dataset to form a deduced data classification; and
analyzing other annotative data associated with equivalently classified data in other datasets.
3. The method of claim 1 further comprising:
identifying that the subset of ingested data omits an association to annotative data representing an annotative description.
4. The method of claim 3 further comprising:
associating the subset of the ingested data with the annotative data based on the deduced data classification to identify the inferred attribute that includes data representing an account identifier associated with the identifier, one or more other account identifiers, and one or more data-related activities.
5. The method of claim 1 further comprising:
identifying that the subset of ingested data is associated with a protected dataset;
determining access to the protected dataset is authorized in association with the account identifier; and
forming a second version of the collaborative dataset.
6. The method of claim 1 further comprising:
generating a data signal specifying information for at least one of the account identifiers that accesses data associated with annotative data; and
causing presentation of the information in an activity feed portion of a user interface.
7. The method of claim 1 further comprising:
converting the plurality of datasets from the different data formats at a format converter module associated with the computing system to form an atomized dataset having a specific format.
8. A system comprising:
one or more computer systems including a memory including executable instructions, and a processor that is configured, responsive to executing the instructions, to:
receive ingested data at a computing system including one or more processors configured to execute instructions to implement one or more executable modules, the ingested data representing a plurality of datasets, each dataset having a different data format as ingested into a dataset ingestion controller module;
format a first dataset of the plurality of datasets to form the first dataset in a collaborative data format;
format a second dataset of the plurality of datasets to form the second dataset in the collaborative data format;
implement the one or more executable modules in a collaborative dataset consolidation system configured to form a collaborative dataset based on the collaborative data format;
determine attributes of the collaborative dataset, a first attribute being associated with an account identifier and a second attribute identifying a first version of the collaborative dataset;
interpret a subset of ingested data of the dataset against one or more data classifications at an inference engine module to derive at least an inferred attribute for the subset of data;
update the collaborative dataset to link to data associated with the ingested data;
transmit activity feed data to disseminate an update to a community of networked users associated with the account identifier, the update including data representing generation of the first version of the collaborative dataset.
9. The system of claim 8 wherein the processor is further configured to:
generate a second version of the collaborative dataset responsive to the link to the data associated with the ingested data including the inferred attribute;
deduce a data classification for the subset of ingested data of the dataset to form a deduced data classification; and
analyze other annotative data associated with equivalently classified data in other datasets.
10. The system of claim 8 wherein the processor is further configured to:
identify that the subset of ingested data omits an association to annotative data representing an annotative description.
11. The system of claim 10 wherein the processor is further configured to:
associate the subset of the ingested data with the annotative data based on the deduced data classification to identify the inferred attribute that includes data representing an account identifier associated with the identifier, one or more other account identifiers, and one or more data-related activities.
12. The system of claim 8 wherein the processor is further configured to:
identify that the subset of ingested data is associated with a protected dataset;
determine access to the protected dataset is authorized in association with the account identifier; and
form a second version of the collaborative dataset.
13. The system of claim 8 wherein the processor is further configured to:
generate a data signal specifying information for at least one of the account identifiers that accesses data associated with annotative data; and
cause presentation of the information in an activity feed portion of a user interface.
14. The system of claim 8 wherein the processor is further configured to:
convert the plurality of datasets from the different data formats at a format converter module associated with the computing system to form an atomized dataset having a specific format.
15. A computer readable storage device comprised of data representing computer instructions configured to cause a processor to initiate a computer-implemented method comprising:
receiving ingested data at a computing system including one or more processors configured to execute instructions to implement one or more executable modules, the ingested data representing at least one dataset of a plurality of datasets, each dataset having a different data format as ingested into a dataset ingestion controller module;
formatting a first dataset of the plurality of datasets to form the first dataset in a collaborative data format;
formatting a second dataset of the plurality of datasets to form the second dataset in the collaborative data format;
implementing the one or more executable modules in a collaborative dataset consolidation system configured to form a collaborative dataset based on the collaborative data format;
determining attributes of the collaborative dataset, a first attribute being associated with an account identifier and a second attribute identifying a first version of the collaborative dataset;
interpreting a subset of ingested data of the dataset against one or more data classifications at an inference engine module to derive at least an inferred attribute for the subset of data;
updating the collaborative dataset to link to data associated with the ingested data;
transmitting activity feed data to disseminate an update to a community of networked users associated with the account identifier, the update including data representing generation of the first version of the collaborative dataset.
16. The computer-implemented of claim 15 further comprising:
generating a second version of the collaborative dataset responsive to the link to the data associated with the ingested data including the inferred attribute;
deducing a data classification for the subset of ingested data of the dataset to form a deduced data classification; and
analyzing other annotative data associated with equivalently classified data in other datasets.
17. The computer-implemented method of claim 15 further comprising:
identifying that the subset of ingested data omits an association to annotative data representing an annotative description.
18. The computer-implemented method of claim 17 further comprising:
associating the subset of the ingested data with the annotative data based on the deduced data classification to identify the inferred attribute that includes data representing an account identifier associated with the identifier, one or more other account identifiers, and one or more data-related activities.
19. The computer-implemented method of claim 15 further comprising:
identifying that the subset of ingested data is associated with a protected dataset;
determining access to the protected dataset is authorized in association with the account identifier; and
forming the second version of the collaborative dataset.
20. The computer-implemented method of claim 15 further comprising:
generating a data signal specifying information for at least one of the account identifiers that accesses data associated with annotative data; and
causing presentation of the information in an activity feed portion of a user interface.