Data set virtual neighborhood characterization, provisioning and access
An apparatus comprises a processing platform configured for characterization, provisioning and access of data set virtual neighborhoods. The processing platform comprises a virtual neighborhood describer, a virtual neighborhood provisioner and a virtual neighborhood visitor. The virtual neighborhood describer is configured to generate characterizations of data set virtual neighborhoods responsive to one or more requirements, such as specified or derived requirements, or combinations thereof. The virtual neighborhood provisioner is configured to utilize the characterizations to generate provisioned data set virtual neighborhoods responsive to one or more corresponding provisioning requests. The virtual neighborhood visitor is configured to access at least one of: (a) one or more data set content indexes associated with at least one of the provisioned data set virtual neighborhoods, and (b) one or more data sets associated with at least one of the provisioned data set virtual neighborhoods, responsive to a given submitted access request.
1. An apparatus comprising:
a processing platform configured for characterization, provisioning and access of data set virtual neighborhoods;
wherein the processing platform comprises:
a virtual neighborhood describer configured to generate characterizations of data set virtual neighborhoods responsive to one or more requirements;
a requirements analyzer associated with the virtual neighborhood describer and adapted to receive the one or more requirements via an interface;
wherein the requirements analyzer processes the one or more requirements and provides information derived therefrom to the virtual neighborhood describer for use in characterizing the data set virtual neighborhoods;
wherein the virtual neighborhood describer in generating the characterizations is further configured to map the one or more requirements to one or more of the data set virtual neighborhoods;
a virtual neighborhood provisioner configured to utilize the characterizations to generate provisioned data set virtual neighborhoods responsive to one or more corresponding provisioning requests; and
a virtual neighborhood visitor configured to access at least one of:
(a) one or more data set content indexes associated with at least one of the provisioned data set virtual neighborhoods; and
(b) one or more data sets associated with at least one of the provisioned data set virtual neighborhoods,
responsive to a given submitted access request;
wherein a given one of the data set virtual neighborhoods for which a characterization is generated comprises a virtual grouping of a plurality of data sets from multiple heterogeneous data sources previously determined to be interrelated based at least in part on metadata describing the data sets, content indexes related to the data sets and content of the data sets;
wherein the virtual neighborhood provisioner is configured to generate a given provisioned data set virtual neighborhood for the given data set virtual neighborhood by setting up, enabling and verifying access to the interrelated data sets of the given data set virtual neighborhood within a context of a data consumer associated with a corresponding provisioning request and
wherein the processing platform further comprises one or more processing devices each comprising a processor coupled to a memory.
2. The apparatus of claim 1 wherein the processing platform further comprises:
a virtual neighborhood rule base adapted for communication with at least a subset of the virtual neighborhood describer, the virtual neighborhood provisioner and the virtual neighborhood visitor;
wherein rules stored in the virtual neighborhood rule base are utilized in at least one of the following:
by the virtual neighborhood describer in generating the characterizations of the data set virtual neighborhoods;
by the virtual neighborhood provisioner in generating the provisioned data set virtual neighborhoods; and
by the virtual neighborhood visitor in accessing at least one of the one or more data set content indexes and the one or more data sets.
3. The apparatus of claim 1 wherein the processing platform further comprises:
a plurality of data set storage devices; and
a plurality of data set processors coupled to the data set storage devices;
wherein the data set processors are configured to process data sets from the data set storage devices to generate at least a portion of the data set content indexes for use by at least one of the virtual neighborhood describer, the virtual neighborhood provisioner and the virtual neighborhood visitor.
4. The apparatus of claim 3 wherein the plurality of data set processors comprise one or more operational processors and one or more informational processors, including at least one of a data set reader, a data set writer, a data set classifier, a data set crawler and a data set indexer.
5. The apparatus of claim 1 wherein the virtual neighborhood describer is configured to generate data set virtual neighborhood descriptors for respective ones of the data set virtual neighborhoods.
6. The apparatus of claim 5 wherein the data set virtual neighborhood descriptors are part of a data set catalog accessible to at least a subset of the virtual neighborhood describer, the virtual neighborhood provisioner and the virtual neighborhood visitor.
7. The apparatus of claim 6 wherein the data set catalog further comprises one or more of data set entries, data set tags, data set classifications, data set interrelationships and data set access control lists.
8. The apparatus of claim 1 wherein the virtual neighborhood provisioner in generating the provisioned data set virtual neighborhoods is further configured to ensure that the provisioned data set virtual neighborhoods satisfy at least one of access control requirements and compliance requirements associated with said one or more provisioning requests.
9. The apparatus of claim 1 wherein the virtual neighborhood visitor in accessing at least one of said one or more data set content indexes and said one or more data sets associated with at least one of the provisioned data set virtual neighborhoods is further configured to integrate, materialize or move at least a subset of said one or more data sets.
10. The apparatus of claim 1 wherein the virtual neighborhood visitor in accessing at least one of said one or more data set content indexes and said one or more data sets associated with at least one of the provisioned data set virtual neighborhoods is further configured to support federated query processing through at least one of: external table mechanisms associated with one or more data management systems, and data virtualization software.
11. The apparatus of claim 1 wherein the given submitted access request comprises one or more of a search request, a query request and a view request.
12. An information processing system comprising the apparatus of claim 1 .
13. A method comprising:
generating characterizations of data set virtual neighborhoods responsive to one or more requirements;
wherein generating characterizations of data set virtual neighborhoods responsive to one or more requirements comprises:
receiving the one or more requirements via an interface;
processing the one or more requirements to provide information derived therefrom for use in characterizing the data set virtual neighborhoods; and
mapping the one or more requirements to one or more of the data set virtual neighborhoods;
utilizing the characterizations to generate provisioned data set virtual neighborhoods responsive to one or more corresponding provisioning requests; and
accessing at least one of:
(a) one or more data set content indexes associated with at least one of the provisioned data set virtual neighborhoods; and
(b) one or more data sets associated with at least one of the provisioned data set virtual neighborhoods,
responsive to a given submitted access request;
wherein a given one of the data set virtual neighborhoods for which a characterization is generated comprises a virtual grouping of a plurality of data sets from multiple heterogeneous data sources previously determined to be interrelated based at least in part on metadata describing the data sets, content indexes related to the data sets and content of the data sets;
wherein generating a given provisioned data set virtual neighborhood for the given data set virtual neighborhood comprises setting up, enabling and verifying access to the interrelated data sets of the given data set virtual neighborhood within a context of a data consumer associated with a corresponding provisioning request; and
wherein the generating, utilizing and accessing are performed by a processing platform comprising one or more processing devices.
14. The method of claim 13 wherein generating characterizations of data set virtual neighborhoods comprises generating data set virtual neighborhood descriptors for respective ones of the data set virtual neighborhoods.
15. The method of claim 14 further comprising storing the data set virtual neighborhood descriptors in a data set catalog.
16. An article of manufacture comprising a processor-readable storage medium having one or more software programs embodied therein, wherein the one or more software programs when executed by at least one processing device of a processing platform cause the processing device:
to generate characterizations of data set virtual neighborhoods responsive to one or more requirements;
wherein generating characterizations of data set virtual neighborhoods responsive to one or more requirements comprises:
receiving the one or more requirements via an interface;
processing the one or more requirements to provide information derived therefrom for use in characterizing the data set virtual neighborhoods; and
mapping the one or more requirements to one or more of the data set virtual neighborhoods;
to utilize the characterizations to generate provisioned data set virtual neighborhoods responsive to one or more corresponding provisioning requests; and
to access at least one of:
(a) one or more data set content indexes associated with at least one of the provisioned data set virtual neighborhoods; and
(b) one or more data sets associated with at least one of the provisioned data set virtual neighborhoods,
responsive to a given submitted access request;
wherein a given one of the data set virtual neighborhoods for which a characterization is generated comprises a virtual grouping of a plurality of data sets from multiple heterogeneous data sources previously determined to be interrelated based at least in part on metadata describing the data sets, content indexes related to the data sets and content of the data sets; and
wherein generating a given provisioned data set virtual neighborhood for the given data set virtual neighborhood comprises setting up, enabling and verifying access to the interrelated data sets of the given data set virtual neighborhood within a context of a data consumer associated with a corresponding provisioning request.
17. The article of manufacture of claim 16 wherein generating characterizations of data set virtual neighborhoods comprises generating data set virtual neighborhood descriptors for respective ones of the data set virtual neighborhoods.
18. The article of manufacture of claim 17 wherein the data set virtual neighborhood descriptors are stored in a data set catalog.
19. The apparatus of claim 1 wherein the virtual grouping of the plurality of data sets of the given data set virtual neighborhood is associated with a sliding time window and further comprises at least one additional data set that is not fully known until after a start time of a current instance of the sliding time window, the given data set virtual neighborhood thereby exhibiting delayed binding to the at least one additional data set.
20. The apparatus of claim 1 wherein the virtual neighborhood visitor is configured:
to authorize access by the data consumer to each of the interrelated data sets of the given data set virtual neighborhood; and
responsive to authorizing the data consumer, to attach a view of the one or more data set context indexes and the one or more data sets associated with the given provisioned data set virtual neighborhood to an analytic sandbox associated with the data consumer.
21. The apparatus of claim 1 wherein at least one of data sets in the generated given provisioned data set virtual neighborhood is selected from the plurality of data sets in the virtual grouping of the given data set virtual neighborhood based on an identity of the data consumer associated with the corresponding provisioning request.
22. The method of claim 13 , further comprising
authorizing access by the data consumer to each of the interrelated data sets of the given data set virtual neighborhood; and
responsive to authorizing the data consumer, attaching a view of the one or more data set context indexes and the one or more data sets associated with the given provisioned data set virtual neighborhood to an analytic sandbox associated with the data consumer.
23. The article of manufacture of claim 16 wherein the one or more software programs when executed further cause the processing device:
to authorize access by the data consumer to each of the interrelated data sets of the given data set virtual neighborhood; and
responsive to authorizing the data consumer, to attach a view of the one or more data set context indexes and the one or more data sets associated with the given provisioned data set virtual neighborhood to an analytic sandbox associated with the data consumer.