Systems and methods for dataset quality quantification in a zero-trust computing environment
Systems and methods for the quantification of sample set quality is provided. In some embodiments, a sample dataset and a sample vector set are received. A rule-based screening of the sample dataset is applied to generate a heuristic quality score. Additionally, a sample vector set is generated from the sample dataset. The difference between the sample vector set and the example vector set is calculated to generate a degree of difference quality score. The heuristic quality score and the degree of difference quality score are normalized and then combined into a quality metric. Calculating the difference between the sample vector set and the example vector set is by framing the distance as a p-value in a hypothesis test, compared against a threshold.
1. A computerized method for dataset quality quantification in a zero-trust environment, the method comprising:
receiving a sample dataset from a data steward;
applying a rule based screening of the sample dataset to generate a heuristic quality score, wherein the heuristic quality score quantifies erroneous data within the sample dataset;
generating a sample vector set from the sample dataset;
receiving an example vector set generated from at least one of a synthetic dataset and an amalgamation of different vector sets from a plurality of different data stewards;
calculating a difference between the sample vector set and the example vector set to generate a degree of difference quality score; and
combining the heuristic quality score and the degree of difference quality score into a quality metric;
determining that the quality metric is above a threshold, and
when the quality metric is above the threshold then recommending the sample dataset to a data consumer.
2. The method of claim 1 , wherein the heuristic quality score and the degree of difference quality score are normalized before combining.
3. The method of claim 1 , wherein the combining is by at least one of adding, averaging, weighted averaging, and multiplying.
4. The method of claim 1 , wherein the heuristic quality score and the degree of difference quality score are bucketized by thresholds before combining.
5. The method of claim 1 , wherein the calculating the difference is by framing the distance as a p-value in a hypothesis test, compared against a threshold.
6. The method of claim 1 , wherein the generating the sample vector set includes:
encoding the dataset according to a set of classes;
generating a matrix of the encoded dataset, wherein each row of the matrix is a patient, and each column is a class or subset of classes in the set of classes; and
converting the generated matrix into a series of vector spaces.
7. A zero-trust computing system for dataset quality quantification, the system comprising:
a data store hosted within a secure computing node for receiving a sample dataset from a data steward, and receiving an example vector set generated from at least one of a synthetic dataset and an amalgamation of different vector sets from a plurality of different data stewards; and
a processor and memory implementing a runtime server within the secure computing node for applying a rule based screening of the sample dataset to generate a heuristic quality score, wherein the heuristic quality score quantifies erroneous data within the sample dataset, generating a sample vector set from the sample dataset, calculating a difference between the sample vector set and the example vector set to generate a degree of difference quality score, and combining the heuristic quality score and the degree of difference quality score into a quality metric, and when the quality metric is above a threshold then recommending the sample dataset to a data consumer.
8. The system of claim 7 , wherein the heuristic quality score and the degree of difference quality score are normalized before combining.
9. The system of claim 7 , wherein the combining is by at least one of adding, averaging, weighted averaging, and multiplying.
10. The system of claim 7 , wherein the heuristic quality score and the degree of difference quality score are bucketized by thresholds before combining.
11. The system of claim 7 , wherein the calculating the difference is by framing the distance as a p-value in a hypothesis test, compared against a threshold.
12. The system of claim 7 , wherein the generating the sample vector set includes:
encoding the dataset according to a set of classes;
generating a matrix of the encoded dataset, wherein each row of the matrix is a patient, and each column is a class or subset of classes in the set of classes; and
converting the generated matrix into a series of vector spaces.