Aggregation of ancillary data associated with source data in a system of networked collaborative datasets
Various embodiments relate generally to data science and data analysis, computer software and systems, and, more specifically, to a computing and data storage platform that facilitates consolidation of one or more datasets, whereby logic is configured to remediate anomalies in a data set originating in a first format prior to enrichment and conversion into a second format that facilitates forming collaborative dataset and, for example, interrelations among a system of networked collaborative datasets, whereby, at least in some implementations, data interrelations between different formats may be disposed in one or more data layers (e.g., layered data files and/or data arrangements). In some examples, a method may converting a dataset from a data format at a format converter to form an atomized dataset in a graph data arrangement, the atomized dataset being a collaborative dataset including atomized descriptor data and atomized source data.
1. A method comprising:
receiving data representing a dataset having a data format into a dataset ingestion controller configured to form a collaborative dataset;
analyzing a subset of the data to determine dataset attributes;
generating descriptor data based on the dataset attributes associated with the subset of the data;
converting the dataset from the data format at a format converter to form an atomized dataset in a graph data arrangement, the atomized dataset being the collaborative dataset including atomized descriptor data and atomized source data, the atomized descriptor data being implemented as a first triple data point and the atomized source data being implemented as a second triple data point;
associating a unit of the descriptor data to a corresponding unit of supra-descriptor data to form associations;
forming another graph data arrangement including the supra-descriptor data and the associations to the descriptor data,
wherein the another graph data arrangement includes pointers to a plurality of atomized collaborative datasets.
2. The method of claim 1 wherein another graph data arrangement excludes the data of the dataset.
3. The method of claim 1 further comprising:
associating the descriptor data to a unit of data representing a state of authorized access.
4. The method of claim 3 further comprising:
receiving a request to access the another graph data arrangement from a computing device associated with a user identifier;
determining permissions associated with the user identifier; and
ensuring the state of authorized access for each unit of the descriptor data facilitates access by the computing device based on the permissions.
5. The method of claim 4 further comprising:
granting access to a portion of the another graph data arrangement.
6. The method of claim 1 further comprising:
revising data representing the supra-dataset attribute descriptor data in a supra-dataset descriptor data repository to link the descriptor data of the collaborative dataset to other descriptor data for other collaborative datasets.
7. The method of claim 1 further comprising:
receiving a request to query the supra-descriptor data; and
performing the query to return resultant data based on the supra-descriptor data.
8. The method of claim 1 wherein converting the dataset to form the atomized dataset in the graph data arrangement comprises:
generating referential data to link a dataset attribute to the subset of data.
9. The method of claim 8 wherein generating the referential data comprises:
linking an first addressable identifier of the dataset attribute to an second addressable identifier of the subset of data.
10. The method of claim 1 wherein associating the descriptor data to supra-dataset attribute descriptor data comprises:
linking descriptor data for a dataset attribute to descriptor data for a supra-dataset attribute.
11. The method of claim 1 wherein analyzing the data to determine dataset attributes comprises:
determining correlated attributes of dataset originating at multiple disparate data sources.
12. An apparatus comprising:
a memory including executable instructions; and
a processor, responsive to executing the instructions, is configured to:
receive data representing a dataset having a data format into a dataset ingestion controller configured to form a collaborative dataset;
analyze a subset of the data to determine dataset attributes;
generate descriptor data based on the dataset attributes associated with the subset of the data;
convert the dataset from the data format at a format converter to form an atomized dataset in a graph data arrangement, the atomized dataset being the collaborative dataset;
associate a unit of the descriptor data to a unit of supra descriptor data to form associations, wherein the unit of the descriptor data is implemented as a first triple data point and the unit of the supra descriptor data is implemented as a second triple data point; and
form another graph data arrangement including the supra descriptor data and the associations to the descriptor data,
wherein the another graph data arrangement includes pointers to a plurality of atomized collaborative datasets.
13. The apparatus of claim 12 wherein the another graph data arrangement excludes the data of the dataset.
14. The apparatus of claim 1 the processor further configured to:
associate the descriptor data to a unit of data representing a state of authorized access.
15. The apparatus of claim 14 wherein the processor is further configured to:
receive a request to access the another graph data arrangement from a computing device associate with a user identifier;
determine permissions associated with the user identifier; and
ensure the state of authorized access for each unit of the descriptor data facilitates access by the computing device based on the permissions.
16. The apparatus of claim 15 the processor further configured to:
grant access to a portion of the another graph data arrangement.