Systems and methods of precision sharing of big data
An ecosystem that allows fine-grained multi-party control over access to information stored in one or more data sources of a data provider. A requesting party can submit a query job to the data provider. Resellers in the chain can introduce their own query modifiers to the query job, adding additional data access, data transformation and segmentation functions to the query job. The data provider can append its own query modifier to the query job and execute the query job with all of the query modifiers. Access control for each link in the chain is checked before the query modifier for the link is executed. After execution of all query modifiers and the query job, the results can be provided to the requesting party.
1. At least one non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to:
receive, from a requestor, a requestor query job comprising: a requestor query step comprising a requestor map step and a requestor reduce step; and a set of requestor attributes, wherein the set of requestor attributes defines access permission with respect to one or more data sources;
identify a data source based on the set of requestor attributes;
prefix the requestor query job with a query modifier associated with the identified data source, the query modifier comprising: a requestor filter; and a data source query modifier step comprising a data source map step and a data source reduce step;
verify that the requestor is authorized to access data elements from the data source based on the requestor filter and set of requestor attributes;
execute, in response to verifying the requestor, the data source map step and the data source reduce step on the data elements from the data source to produce a requestor modifier output;
execute the requestor map step and the requestor reduce step using the requestor modifier output to produce a requestor output;
wherein the requestor attributes includes a certified attribute, and wherein the verify instruction comprises verifying that the requestor is authorized to access data elements from the data source based on the certified attribute and the requestor filter.
2. The at least one non-transitory computer-readable storage medium of claim 1 , further comprising instructions that cause the at least one processor to: generate a result view based on the requestor output; and provide the result view to the requestor.
3. The at least one non-transitory computer-readable storage medium of claim 1 , wherein: the requestor query step in the requestor query job comprises a requestor query modifier step, the requestor query job further comprising: a second requestor filter; and a second requestor query job corresponding to a second requestor, the second requestor query job comprising: a second requestor query step corresponding to the second requestor; a second set of requestor attributes corresponding to the second requestor; and the instructions further cause the at least one processor to: verify, after producing the requestor output, that the second requestor is authorized to access the data from the data source according to the requestor output based on the second requestor filter and the second set of requestor attributes; execute the second requestor query step to produce a second requestor output; generate a result view based on the second requestor output; and provide the result view to the second requestor.
4. The at least one non-transitory computer-readable storage medium of claim 3 , wherein the first requestor comprises a reseller and wherein the second requestor comprises a requesting user interface that submitted the second requestor query job.
5. The at least one non-transitory computer-readable storage medium of claim 3 , wherein the data source comprises at least one of a medical data source, a retailer data source, a financial data source, a gaming data source, a government data source, and an educational data source.
6. The at least one non-transitory computer-readable storage medium of claim 1 , wherein the requestor filter comprises a set of access attributes for the data source, and the verify instruction further comprises matching at least a subset of the set requestor attributes with at least a subset of access attributes for the data source.
7. The at least one non-transitory computer-readable storage medium of claim 6 , wherein: the requestor filter comprises a set of metadata attributes for the data source; the verify instruction further comprises matching at least a subset of the set requestor attributes with at least a subset of the metadata attributes for the data source; and wherein the instructions further cause the at least one processor to, in response to verification, allow access to metadata for the data source.
8. The at least one non-transitory computer-readable storage medium of claim 1 , wherein the requestor query step comprises a database query.
9. The at least one non-transitory computer-readable storage medium of claim 1 , wherein: the requestor query step comprises a requestor data access step and at least one of a requestor data segmentation step and a requestor data transformation step; and the data
source query modifier step comprises a data source data access step, and at least one of a data source data segmentation step and a data source data transformation step.
10. The at least one non-transitory computer-readable storage medium of claim 9 , wherein at least one of the requestor query step the data source query modifier step further comprises a runtime limitation step, the runtime limitation step comprising at least one of a runtime duration limitation and a processor load limitation.
11. The at least one non-transitory computer-readable storage medium of claim 1 , wherein: the requestor map step in the requestor query job comprises a requestor modifying map step, the requestor query job further comprising: a second requestor filter step; a second requestor map step corresponding to the second requestor; a second set of requestor attributes corresponding to a second requestor; and wherein the instructions further cause the at least one processor to: verify, after producing the requestor map output, that the second requestor is authorized to access the data from the data source according to the requestor map output based on the second requestor filter step and the second set of requestor attributes; execute the second requestor map step to produce a second requestor map output; execute the reduce step based on the second requestor map output; and return the results of the reduce step to the second requestor.
12. The at least one non-transitory computer-readable storage medium of claim 1 , wherein the requestor query job comprises compiled code.