Data ingestion to generate layered dataset interrelations to form a system of networked collaborative datasets
Various embodiments relate generally to data science and data analysis, and computer software and systems to provide an interface between repositories of disparate datasets and computing machine-based entities that seek access to the datasets, and, more specifically, to a computing and data storage platform that facilitates consolidation of one or more datasets, whereby data ingestion is performed to form data representing layered data files and data arrangements to facilitate, for example, interrelations among a system of networked collaborative datasets. In some examples, a method may include forming a first layer data file and a second layer data file, assigning addressable identifiers to uniquely identify units of data and data units to facilitate the linking of data, and implementing selectively one or more of a unit of data and a data unit as a function of a context of a data access request for a collaborative dataset.
1. A method comprising:
receiving data representing a set of data formatted in a data arrangement;
adapting the data arrangement for the set of data to form a dataset having a first data format;
forming multiple layer data files to link data in a second data format to data in the first format, wherein forming the multiple layer data files comprises:
forming one or more first layer data files to reference a set of data associated with the second data format in which units of data in the set of data are configured to link with other layer data files, and
forming one or more second layer data files that include a subset of data based on the set of data in the second data format, data units of the subset of data in the second data format being configured to link to the units of data in the first data format;
retrieving data representing a data attribute with which to analyze data from the data arrangement;
analyzing a subset of the data to detect a non-compliant data attribute;
invoking an action responsive to the non-compliant data attribute to modify the subset of data to form a modified subset of the data directed to affecting a condition; and
predicting data specifying at least one link between the dataset and another dataset;
generating a graph data arrangement including links with the modified subset of the data, the graph data arrangement being linkable via the at least one link to other graph data arrangements as a collaborative dataset;
assigning addressable identifiers to uniquely identify the units of data and the data units to facilitate linking data between the set of data in the first format and the dataset in the second data format, at least one of the addressable identifiers referencing a triplestore database; and
implementing selectively one or more of a unit of data and a data unit as a function of a context of a data access request.
2. The method of claim 1 wherein the data in the second data format include at least a portion of data formatted as triples.
3. The method of claim 1 further comprising:
detecting the condition based on the non-compliant data attribute for the subset of data.
4. The method of claim 1 further comprises:
generating data to present one or more user inputs in a user interface to initiate execution of instructions to resolve the condition; and
receiving a response to at least one of the one or more user inputs to invoke the action.
5. The method of claim 4 wherein retrieving the data representing the data attribute comprises:
receiving data representing a configurable data attribute for the subset of data, the configurable data attribute including a configurable data property and a configurable value with which to analyze data from the data arrangement.
6. The method of claim 1 wherein analyzing to detect the non-compliant data attribute further comprises:
identifying a data characteristic specifying a portion of the subset of data as including restricted data.
7. The method of claim 1 wherein analyzing to detect the non-compliant data attribute further comprises:
characterizing the subset of data to identify a pattern of data; and
matching the pattern of data against the data fields to identify a noncompliant portion of the data.
8. A system comprising:
a memory including executable instructions; and
a processor, responsive to executing the instructions, is configured to:
receive data representing a set of data formatted in a data arrangement;
adapt the data arrangement for the set of data to form a dataset having a first data format;
form multiple layer data files to link data in a second data format to data in the first format, wherein the processor is further configured to:
form one or more first layer data files to reference a set of data associated with the second data format in which units of data in the set of data are configured to link with other layer data files, and
form one or more second layer data files that include a subset of data based on the set of data in the second data format, data units of the subset of data in the second data format being configured to link to the units of data in the first data format;
retrieve data representing a data attribute with which to analyze data from the data arrangement;
analyze a subset of the data to detect a non-compliant data attribute;
invoke an action responsive to the non-compliant data attribute to modify the subset of data to form a modified subset of the data directed to affecting a condition; and
predict data specifying at least one link between the dataset and another dataset;
generate a graph data arrangement including links with the modified subset of the data, the graph data arrangement being linkable via the at least one link to other graph data arrangements as a collaborative dataset;
assign addressable identifiers to uniquely identify the units of data and the data units to facilitate linking data between the set of data in the first format and the dataset in the second data format, at least one of the addressable identifiers referencing a triplestore database; and
implement selectively one or more of a unit of data and a data unit as a function of a context of a data access request.
9. The system of claim 8 wherein the data in the second data format include at least a portion of data formatted as triples.
10. The system of claim 8 wherein the processor is further configured to:
detect the condition based on the non-compliant data attribute for the subset of data.
11. The system of claim 8 wherein the processor is further configured to:
generate data to present one or more user inputs in a user interface to initiate execution of instructions to resolve the condition; and
receive a response to at least one of the one or more user inputs to invoke the action.
12. The system of claim 11 wherein the processor is further configured to:
receive data representing a configurable data attribute for the subset of data, the configurable data attribute including a configurable data property and a configurable value with which to analyze data from the data arrangement.
13. The system of claim 8 wherein the processor is further configured to:
identify a data characteristic specifying a portion of the subset of data as including restricted data.
14. The system of claim 8 wherein the processor is further configured to:
characterize the subset of data to identify a pattern of data; and
match the pattern of data against the data fields to identify a noncompliant portion of the data.