Method and system for generating a unified database from data sets

1. A computer-implemented method for generating a unified clinical database, comprising:

receiving a structured set of clinical data, the set comprising one or more records, each record having one or more fields;

aggregating values taken from a first field of the records;

calculating a statistical distribution of the aggregated first field values;

comparing the statistical distribution of the aggregated first field values with statistical distributions of stored data measures;

calculating a confidence level regarding a label for the first field;

if the confidence level meets or exceeds a pre-determined threshold,

providing the label to the first field; and

storing the first field values in a first unified field in the unified database; and

if the confidence level does not exceed the pre-determined threshold, receiving more information and recalculating the confidence level,

wherein providing the label to the first field comprises determining a data measure having a statistical distribution closest to the statistical distribution of the aggregated first field values.

2. The method of claim 1 , further comprising calculating the statistical distributions of the stored data measures.

3. The method of claim 1 , further comprising applying to the aggregated first field values a set of rules including eligibility criteria comprising inclusion criteria, exclusion criteria, or both.

4. A computer-implemented method for generating a unified clinical database, comprising:

receiving a structured set of clinical data, the set comprising one or more records, each record having two or more fields;

aggregating values taken from a first field of the records;

aggregating values taken from a second field of the records;

calculating statistical distributions of the aggregated first and second field values;

comparing the statistical distributions of the aggregated first and second field values with statistical distributions of stored data measures;

automatically applying, by a processor, a set of rules to the aggregated first and second field values to determine correlations between the aggregated first and second field values; and

determining labels for the aggregated first and second field values based on the rules and the correlations, by:

determining a confidence level regarding a label for the first field;

if the confidence level meets or exceeds a pre-determined threshold, providing the label to the first field; and

storing the first field values in a first unified field in the unified database;

and

if the confidence level does not exceed the pre-determined threshold, receiving more information and recalculating the confidence level.

5. The method of claim 4 , wherein the more information comprises information regarding the correlations between the aggregated first and second field values.

6. The method of claim 4 , wherein determining labels for the aggregated first and second field values comprises:

calculating a second confidence level regarding a label for the second field;

if the second confidence level meets or exceeds a pre-determined threshold, providing the label to the second field; and

storing the second field values in a second unified field in the unified database;

and

if the second confidence level does not exceed the pre-determined threshold, receiving more information and recalculating the second confidence level.

7. The method of claim 6 , wherein the more information comprises information regarding the correlations between the aggregated first and second field values.

8. The method of claim 4 , wherein determining labels for the aggregated first and second field values comprises determining data measures having statistical distributions closest to the statistical distributions of the aggregated first and second field values.

9. The method of claim 4 , wherein applying the set of rules to the aggregated first and second field values comprises determining a structure of the records.

10. The method of claim 9 , wherein the structure of the records of a CHEM-7 test comprises seven fields.

11. The method of claim 4 , wherein determining a confidence level regarding a label for the first field comprises calculating the confidence level.

12. A computer-implemented method for generating a unified clinical database, comprising:

receiving a structured set of clinical data, the set comprising one or more records, each record having one or more fields;

aggregating values taken from a first field of the records;

automatically applying, by a processor, a set of rules to the aggregated first field values,

wherein applying the set of rules includes comparing the aggregated first field values to:

stored statistics of known clinical measures, if the aggregated first field values comprise numerical values;

known text entries from a clinical dictionary, if the aggregated first field values comprise alphabetical and alphanumeric values; and

stored calendar information, if the aggregated first field values comprise date values;

calculating a confidence level regarding a label for the first field;

if the confidence level meets or exceeds a pre-determined threshold, providing the label to the first field; and

storing the first field values in a first unified field in the unified database; and

if the confidence level does not exceed the pre-determined threshold, receiving more information and recalculating the confidence level.

13. The method of claim 12 , wherein the rules include eligibility criteria comprising inclusion criteria, exclusion criteria, or both.

14. The method of claim 12 , wherein the rules are refined based on the received data.

15. The method of claim 12 , wherein the stored statistics comprise statistical distributions of known clinical measures.

16. The method of claim 12 , wherein if the aggregated first field values comprise date values, and the date values are after the beginning of a clinical trial, then the first field label refers to testing during the clinical trial.

17. The method of claim 12 , wherein if the aggregated first field values comprise date values, and the date values are before the beginning of a clinical trial, then the first field label refers to historic data.

18. The method of claim 12 , further comprising:

aggregating values taken from a second field of the records;

automatically applying the set of rules to the aggregated second field values to determine correlations between the aggregated first and second field values; and

determining a label for the second field values.

19. The method of claim 18 , wherein said correlations increase the confidence level regarding the label for the first field values.