Centralized repository and data sharing hub for establishing model sufficiency

1 . A system for centralized transfer of model input data, the system comprising:

a hardware processor; and

a memory storing instructions which when executed by the hardware processor, cause the hardware processor to:

receive a dataset including data that is suitable to use as input to model behavior and for subsequent representation on a sufficiency listing;

modify the sufficiency listing to include a representation of the dataset;

generate a plurality of sufficiency parameters for the dataset in a sufficiency database, the sufficiency database comprising a plurality of sufficiency parameters for a plurality of other datasets;

generate, by an artificial intelligence program comprising a front-end neural network configured to reduce a dimensionality of the data of the dataset and a back-end neural network configured to receive the reduced dimensionality data from the front-end neural network and perform statistical processing on the reduced dimensionality data, a dataset sufficiency-indicator for the dataset based on: (i) the data of the dataset, (ii) the generated plurality of sufficiency parameters for the dataset, and (iii) the plurality of sufficiency parameters of the other datasets, wherein the artificial intelligence program is trained based on the plurality of sufficiency parameters for the plurality of other datasets, wherein the plurality of sufficiency parameters of the other datasets comprise:

a respective actual transaction value of the other dataset,

a respective size of the other dataset,

a respective elapsed time since transfer of the other dataset,

a respective granularity of the data in the other dataset,

a respective retention time for the other dataset, and

a respective population represented by the other dataset;

generate a metadata tag that encodes the dataset sufficiency-indicator;

modify the sufficiency listing to include the metadata tag in association with the representation of the dataset;

store the metadata tag in association with the dataset in the sufficiency database;

in response to determining that the dataset sufficiency-indicator for the dataset is below a sufficiency threshold, receive additional data comprising: (i) determined sufficiency parameters for the sufficiency database, and (ii) external sufficiency parameters associated with an external dataset, the external sufficiency parameters comprising a real-time sufficiency adjustment;

retrain the artificial intelligence program based on the additional data by adjusting parameters of the front-end neural network and the back-end neural network to maintain reduced output error for dataset sufficiency-indicators;

generate, by the retrained artificial intelligence program, another dataset sufficiency-indicator for the dataset;

update the metadata tag stored in the sufficiency database to encode the another dataset sufficiency-indicator;

determine that the another dataset sufficiency-indicator is above the sufficiency threshold; and

based on the determination that the another dataset sufficiency-indicator is above the sufficiency threshold, communicate, via a network, a notification comprising an indication that the another dataset sufficiency-indicator is above the sufficiency threshold.

2 . The system of claim 1 , wherein the instruction to connect the dataset sufficiency-indicator to the dataset includes to modify the sufficiency listing to include the dataset sufficiency-indicator in association with the representation of the dataset.

3 . The system of claim 1 , wherein the instruction to connect the dataset sufficiency-indicator to the dataset includes to associate a tag with the dataset representative of the dataset sufficiency-indicator.

4 . The system of claim 1 , wherein the data of the dataset comprises synthetic data.

5 . The system of claim 1 , wherein the instructions further cause the hardware processor to:

generate, from the data of the dataset, a synthetic dataset including synthetic data suitable for a subsequent transfer.

6 . The system of claim 1 , wherein the real-time sufficiency adjustment includes at least one inflation indicator, a consumer confidence indicator, a consumer sentiment indicator, a size of the external dataset, a time-period since a transfer of the external dataset, a granularity of data of the external dataset, a retention time before the transfer of the external dataset, a population associated with the external dataset, a sufficiency indicator for the external dataset, or an actual sufficiency for the transfer of the external dataset.

7 . The system of claim 1 , wherein the artificial intelligence program comprises a machine learning algorithm.

8 . The system of claim 1 , wherein the instructions further cause the hardware processor to:

receive a sufficiency purge value; and

generate a communication when the dataset sufficiency-indicator is greater than the sufficiency purge value.

9 . The system of claim 1 , wherein the additional data further comprises a synthetic dataset including synthetic data.

10 . The system of claim 9 , wherein the instructions, when executed by the hardware processor, cause the hardware processor to:

generate the synthetic dataset including the synthetic data based on the dataset.

11 . A system non-transitory computer-readable medium having stored thereon instructions, which when executed by a hardware processor, cause the hardware processor to:

receive a dataset including data that is suitable to use as input to model behavior and for subsequent representation on a sufficiency listing;

modify the sufficiency listing to include a representation of the dataset;

generate a plurality of sufficiency parameters for the dataset in a sufficiency database, the sufficiency database comprising a plurality of sufficiency parameters for a plurality of other datasets;

generate, by an artificial intelligence program comprising a front-end neural network configured to reduce a dimensionality of the data of the dataset and a back-end neural network configured to receive the reduced dimensionality data from the front-end neural network and perform statistical processing on the reduced dimensionality data, a dataset sufficiency-indicator for the dataset based on: (i) the data of the dataset, (ii) the generated plurality of sufficiency parameters for the dataset, and (iii) the plurality of sufficiency parameters of the other datasets, wherein the artificial intelligence program is trained based on the plurality of sufficiency parameters for the plurality of other datasets, wherein the plurality of sufficiency parameters of the other datasets comprise:

a respective actual transaction value of the other dataset,

a respective size of the other dataset,

a respective elapsed time since transfer of the other dataset,

a respective granularity of the data in the other dataset,

a respective retention time for the other dataset, and

a respective population represented by the other dataset;

generate a metadata tag that encodes the dataset sufficiency-indicator;

modify the sufficiency listing to include the metadata tag in association with the representation of the dataset;

store the metadata tag in association with the dataset in the sufficiency database;

in response to determining that the dataset sufficiency-indicator for the dataset is below a sufficiency threshold, receive additional data comprising: (i) determined sufficiency parameters for the sufficiency database, and (ii) external sufficiency parameters associated with an external dataset, the external sufficiency parameters comprising a real-time sufficiency adjustment;

retrain the artificial intelligence program based on the additional data by adjusting parameters of the front-end neural network and the back-end neural network to maintain reduced output error for dataset sufficiency-indicators;

generate, by the retrained artificial intelligence program, another dataset sufficiency-indicator for the dataset;

update the metadata tag stored in the sufficiency database to encode the another dataset sufficiency-indicator;

determine that the another dataset sufficiency-indicator is above the sufficiency threshold; and

based on the determination that the another dataset sufficiency-indicator is above the sufficiency threshold, communicate, via a network, a notification comprising an indication that the another dataset sufficiency-indicator is above the sufficiency threshold.

12 . The non-transitory computer-readable medium of claim 11 , wherein the instructions further cause the hardware processor to:

generate at least one sufficiency parameter in the sufficiency database utilizing a previous transfer module.

13 . The non-transitory computer-readable medium of claim 11 , wherein the dataset sufficiency-indicator is an inference of the artificial intelligence program.

14 . The non-transitory computer-readable medium of claim 11 , wherein the additional data further comprises a synthetic dataset including synthetic data.

15 . The non-transitory computer-readable medium of claim 14 , wherein the instructions, when executed by the hardware processor, cause the hardware processor to:

generate the synthetic dataset including the synthetic data based on the dataset.

16 . A method of adding a dataset to a sufficiency listing utilized for the transfer of model input data, the method comprising:

receiving, by a hardware processor, a dataset including data that is suitable to use as input to model behavior and for subsequent representation on the sufficiency listing;

modifying, by the hardware processor, the sufficiency listing to include a representation of the dataset;

generating, by the hardware processor, a plurality of sufficiency parameters for the dataset in a sufficiency database, the sufficiency database comprising a respective plurality of sufficiency parameters for a plurality of other datasets;

generating, by an artificial intelligence program executing on the hardware processor wherein the artificial intelligence program comprises a front-end neural network configured to reduce a dimensionality of the data of the dataset and a back-end neural network configured to receive the reduced dimensionality data from the front-end neural network and perform statistical processing on the reduced dimensionality data, a dataset sufficiency-indicator for the dataset based on (i) the data of the dataset, (ii) the generated plurality of sufficiency parameters for the dataset, and (iii) the plurality of sufficiency parameters of the other datasets, wherein the artificial intelligence program is trained based on the plurality of sufficiency parameters for the plurality of other datasets, wherein the plurality of sufficiency parameters of the other datasets comprise:

a respective actual transaction value of the other dataset,

a respective size of the other dataset,

a respective elapsed time since transfer of the other dataset,

a respective granularity of the data in the other dataset,

a respective retention time for the other dataset, and

a respective population represented by the other dataset;

generating, by the hardware processor, a metadata tag that encodes the dataset sufficiency-indicator;

modifying, by the hardware processor, the sufficiency listing to include the metadata tag in association with the representation of the dataset;

storing, by the hardware processor, the metadata tag in association with the dataset sufficiency in the sufficiency database;

in response to determining, by the hardware processor, that the dataset sufficiency-indicator for the dataset is below a sufficiency threshold, receiving, by the hardware processor, additional data comprising: (i) determined sufficiency parameters for the sufficiency database, and (ii) external sufficiency parameters associated with an external dataset, the external sufficiency parameters comprising a real-time sufficiency adjustment;

retraining, by the hardware processor, the artificial intelligence program based on the additional data by adjusting parameters of the front-end neural network and the back-end neural network to maintain reduced output error for dataset sufficiency-indicators;

generating, by the retrained artificial intelligence program, another dataset sufficiency-indicator for the dataset;

updating, by the hardware processor, the metadata tag stored in the sufficiency database to encode the another dataset sufficiency-indicator;

determining, by the hardware processor, that the another dataset sufficiency-indicator is above the sufficiency threshold; and

based on the determination that the another dataset sufficiency-indicator is above the sufficiency threshold, communicating, by the hardware processor via a network a notification comprising an indication that the another dataset sufficiency-indicator is above the sufficiency threshold.

17 . The method of claim 16 , wherein the additional data further comprises a synthetic dataset including synthetic data.

18 . The method of claim 17 , further comprising:

generating, by the hardware processor, the synthetic dataset including the synthetic data based on the dataset.