Validation of AI models using holdout sets

1. A method for automating model validation, said method comprising:

receiving an original dataset via a processor, wherein said original dataset has at least one linked dataset;

segmenting, automatically, said original dataset into a plurality of data groups, wherein said plurality of data groups include a model training dataset and a holdout dataset, and wherein said holdout dataset maintains referential integrity across linked datasets;

generating a model with said model training dataset by performing one or more repeatable actions on said model training dataset;

triggering automatic model validation of said model by copying said model into a pre-production environment; and

validating said model with said holdout dataset.

2. The method of claim 1 further comprising:

retrieving, by said processor, said model training dataset from said original dataset, wherein said holdout dataset is unavailable for use in said model training dataset;

defining, by a user, said one or more repeatable actions to be performed on said model training dataset via said processor after segmenting said original dataset; and

performing, automatically, said one or more repeatable actions on said holdout dataset.

3. The method of claim 1 further comprising:

selecting said original dataset from a catalog; and

aggregating said original dataset in a project.

4. The method of claim 3 wherein said selecting said original dataset includes receiving user input by a user, wherein said user input includes said user selecting said original dataset.

5. The method of claim 1 further comprising:

comparing validation results of said model to pre-selected metrics, wherein said pre-selected metrics establish a model validation threshold; and

rejecting said model if said model fails to meet said model validation threshold.

6. The method of claim 5 wherein said pre-selected metrics are selected from a list consisting of fairness, bias, quality, and drift.

7. The method of claim 1 wherein segmenting, automatically, said original dataset into a plurality of data groups includes randomly, in a uniform fashion, segmenting said original dataset.

8. A system that automatically validates models, said system comprising:

a memory; and

a processor in communication with said memory, said processor being configured to perform operations comprising:

receiving an original dataset, wherein said original dataset has at least one linked dataset;

segmenting, automatically, said original dataset into a plurality of data groups, wherein said plurality of data groups include a model training dataset and a holdout dataset, wherein said holdout dataset maintains referential integrity across linked datasets;

generating a model with said model training dataset by performing one or more repeatable actions on said model training dataset;

triggering automatic model validation of said model by copying said model into a pre-production environment; and

validating said model with said holdout dataset.

9. The system of claim 8 wherein the operations further comprise:

retrieving, by said processor, said model training dataset from said original dataset, wherein said holdout dataset is unavailable for use in said model training dataset;

defining, by a user, said one or more repeatable actions to be performed on said model training dataset via said processor after segmenting said original dataset; and

performing, automatically, said one or more repeatable actions on said holdout dataset.

10. The system of claim 8 wherein the operations further comprise:

selecting said original dataset from a catalog; and

aggregating said original dataset in a project.

11. The system of claim 10 wherein said selecting said original dataset includes receiving user input by a user, wherein said user input includes said user selecting said original dataset.

12. The system of claim 8 wherein the operations further comprise:

comparing validation results of said model to pre-selected metrics, wherein said pre-selected metrics establish a model validation threshold; and

rejecting said model if said model fails to meet said model validation threshold.

13. The system of claim 8 wherein segmenting, automatically, said original dataset into a plurality of data groups includes randomly, in a uniform fashion, segmenting said original dataset.

14. A computer program product for automatic model validation, said computer program product comprising a computer readable storage medium having program instructions embodied therewith, said program instructions executable by a processor to cause said processor perform a function, said function comprising:

receiving an original dataset, wherein said original dataset has at least one linked dataset;

segmenting, automatically, said original dataset into a plurality of data groups, wherein said plurality of data groups include a model training dataset and a holdout dataset, and wherein said holdout dataset maintains referential integrity across linked datasets;

generating a model with said model training dataset by performing one or more repeatable actions on said model training dataset;

triggering automatic model validation of said model by copying said model into a pre-production environment; and

validating said model with said holdout dataset.

15. The computer program product of claim 14 wherein said function further comprises:

retrieving, by said processor, said model training dataset from said original dataset, wherein said holdout dataset is unavailable for use in said model training dataset;

defining, by a user, said one or more repeatable actions to be performed on said model training dataset via said processor after segmenting said original dataset; and

performing, automatically, said one or more repeatable actions on said holdout dataset.

16. The computer program product of claim 14 wherein said function further comprises:

comparing validation results of said model to pre-selected metrics, wherein said pre-selected metrics establish a model validation threshold, and wherein said pre-selected metrics are selected from a list consisting of fairness, bias, quality, and drift; and

rejecting said model if said model fails to meet said model validation threshold.

17. The computer program product of claim 14 wherein segmenting, automatically, said original dataset into a plurality of data groups includes randomly, in a uniform fashion, segmenting said original dataset.