Collaborative dataset consolidation via distributed computer networks
Various embodiments relate generally to data science and data analysis, computer software and systems, and wired and wireless network communications to provide an interface between repositories of disparate datasets and computing machine-based entities that seek access to the datasets, and, more specifically, to a computing and data storage platform that facilitates consolidation of one or more datasets, whereby a collaborative data layer and associated logic facilitate, for example, efficient access to, and implementation of, collaborative datasets. In some examples, a method may include receiving data representing a query into a collaborative dataset consolidation system, identifying datasets relevant to the query, generating one or more queries to access disparate data repositories, and retrieving data representing query results. In some cases, one or more queries are applied (e.g., as a federated query) to atomized datasets stored in one or more atomized data stores, at least two of which may be different.
1. A method comprising:
receiving a data file including a dataset into a collaborative dataset consolidation system;
formatting the dataset to form a first atomized dataset including atomized data points each including data representing at least two objects and an association between the two objects;
forming a second atomized dataset including the first atomized dataset and one or more other atomized datasets;
receiving data representing a query into the collaborative dataset consolidation system, the query being associated with an identifier;
identifying a subset of the second atomized dataset relevant to the query, wherein portions of the second atomized dataset are disposed in different data repositories;
generating a plurality of sub-queries each of which is configured to access at least one of the different data repositories; and
retrieving data representing query results from the at least one of the different data repositories.
2. The method of claim 1 wherein generating the plurality of sub-queries comprises:
classifying query portions.
3. The method of claim 2 wherein classifying the query portions comprises:
identifying a classification type for a portion of the query.
4. The method of claim 1 wherein the datasets comprise linked data points.
5. The method of claim 4 wherein linked data points comprise triples.
6. The method of claim 5 wherein at least one triple of the triples are formatted to comply with a Resource Description Framework (“RDF”) data model.