Systems and methods for in-situ quality evaluation of sensitive data

1 . A system, comprising:

a processing platform that is implemented in a first environment operated by a data quality evaluation services provider, the processing platform being configured to:

analyze data quality evaluation requirements associated with source data of a first entity that is different from the data quality evaluation services provider,

identify one or more data quality evaluation algorithms based at least in part on an analysis of the data quality evaluation requirements, and

store metadata relating to evaluation of the source data; and

an execution platform that is deployed in a local or cloud-based environment associated with the first entity, the execution platform being segregated from the processing platform by one or more networks, the execution platform being configured to:

receive the one or more data quality evaluation algorithms from the processing platform and execute the one or more data quality evaluation algorithms for the evaluation of the source data to derive data quality metrics, and

provide the data quality metrics to the processing platform for storage as the metadata,

wherein the execution platform performs the evaluation of the source data of the first entity in situ without the source data of the first entity being accessible outside of the execution platform, including to the data quality evaluation services provider through the processing platform, thereby ensuring data integrity, data locality compliance, or both.

2 . The system of claim 1 , wherein the processing platform comprises a metadata store for storing the metadata.

3 . The system of claim 2 , wherein the data quality evaluation services provider also provides the execution platform for deployment in the local or cloud-based environment associated with the first entity.

4 . The system of claim 1 , wherein the processing platform comprises one or more interfaces configured to enable user interaction with the processing platform, and wherein the one or more interfaces comprise one or more web-based portals, one or more application programming interfaces (APIs), or a combination thereof for facilitating user submission of data quality evaluation jobs, user submission of the data quality evaluation requirements, user submission of bring your own code (BYOC) data quality evaluation algorithms, user accessing of the metadata, or a combination thereof.

5 . The system of claim 1 , further comprising a second execution platform that is deployed in a second local or cloud-based environment associated with a second entity different from the first entity and the data quality evaluation services provider, wherein the processing platform analyzes second data quality evaluation requirements associated with second source data of the second entity, identifies one or more second data quality evaluation algorithms based at least in part on an analysis of the second data quality evaluation requirements, and stores second metadata relating to evaluation of the second source data, wherein the second execution platform is segregated from the processing platform by one or more second networks, and wherein the second execution platform receives the one or more second data quality evaluation algorithms from the processing platform, executes the one or more second data quality evaluation algorithms for the evaluation of the second source data to derive second data quality metrics, and provides the second data quality metrics to the processing platform for storage as the second metadata, the processing platform thereby providing in-situ data quality evaluation services for different entities via different deployed execution platforms.

6 . The system of claim 1 , wherein the execution platform comprises an in-situ execution module that executes the one or more data quality evaluation algorithms, one or more bring your own code (BYOC) data quality evaluation algorithms, or a combination thereof.

7 . The system of claim 1 , wherein the execution platform comprises a scalable architecture of application servers for facilitating varying loads.

8 . The system of claim 1 , wherein the execution platform comprises a federated query interface (FQI) that integrates with one or more third-party distributed query engines for executing queries across distributed data systems without transmitting the source data.

9 . The system of claim 1 , wherein the execution platform comprises an interface for facilitating injection of custom functions for use in the evaluation.

10 . The system of claim 1 , wherein the execution platform comprises a caching layer configured to reduce a load on primary data storage systems during the evaluation.

11 . The system of claim 1 , wherein the evaluation is performed in real-time or in batch mode.

12 . The system of claim 1 , wherein the execution platform comprises a transient data storage configured to cache data during the evaluation, enable downloading of the data quality metrics, or a combination thereof.

13 . The system of claim 1 , wherein an executor in the execution platform is configured to submit a registration request to the processing platform upon instantiation of the executor to facilitate registration of the execution platform with the processing platform.

14 . The system of claim 1 , wherein the processing platform is configured to control the evaluation by performing one or more of the following:

facilitating data discovery;

registering metadata;

processing data quality commands; or

a combination thereof.

15 . The system of claim 1 , wherein the one or more networks include an Internet.

16 . A non-transitory machine-readable medium, comprising executable instructions that, when executed by a processing platform including a processor, facilitate performance of operations, the operations comprising:

receiving, from a user device, a request to perform data quality evaluation of source data of a first entity, wherein the source data is located in an environment associated with the first entity that is segregated from the processing platform, and wherein the processing platform is operated by a data quality evaluation services provider that is different from the first entity;

identifying one or more data quality processing algorithms based on the request;

causing the one or more data quality processing algorithms to be provided to an execution platform that is deployed in the environment associated with the first entity, thereby enabling the execution platform to perform the data quality evaluation of the source data of the first entity using the one or more data quality processing algorithms, wherein the execution platform performs the data quality evaluation of the source data of the first entity in situ without the source data of the first entity being accessible outside of the execution platform, including to the data quality evaluation services provider through the processing platform, thereby ensuring data integrity, data locality compliance, or both; and

receiving, from the execution platform, data quality metrics resulting from the data quality evaluation.

17 . The non-transitory machine-readable medium of claim 16 , wherein the operations further comprise:

receiving information regarding at least one custom algorithm to be used in the data quality evaluation; and

causing the information to be provided to the execution platform, thereby enabling the execution platform to utilize the at least one custom algorithm in the data quality evaluation.

18 . The non-transitory machine-readable medium of claim 16 , wherein the data quality evaluation services provider operates the processing platform and provides the execution platform for deployment in the environment associated with the first entity.

19 . A method, comprising:

obtaining, from a processing platform by an execution platform including a processor, one or more data quality processing algorithms to be used for evaluating data quality of source data of a first entity, wherein the processing platform is implemented in a first environment operated by a data quality evaluation services provider, wherein the execution platform is deployed in a local or cloud-based environment associated with the first entity, and wherein the execution platform is segregated from the processing platform by one or more networks;

performing, by the execution platform, a data quality evaluation of the source data using the one or more data quality processing algorithms, resulting in data quality metrics; and

causing, by the execution platform, the data quality metrics to be provided to the processing platform for storage,

wherein the execution platform performs the data quality evaluation of the source data of the first entity in situ without the source data of the first entity being exposed outside of the execution platform, including to the data quality evaluation services provider through the processing platform, thereby enabling data integrity, data locality compliance, or both during the data quality evaluation.

20 . The method of claim 19 , wherein the processing platform identifies the one or more data quality processing algorithms in response to a user request to evaluate the source data.