Intelligent file-level validation
Intelligent file-level validation is described. A service provider (e.g., an administrator) can determine rule(s) based partly on specified data. The service provider can receive a data file. The service provider can determine modification data that indicates an amount of change to result from modifying previously stored data based on the data file, and can analyze the modification data based partly on threshold(s) associated with the rule(s). Each threshold can indicate an amount of change that is determined to be permissible for the data file, for the corresponding rule. The service provider can determine whether the data file is valid based partly on analyzing the modification data in view of the rule(s). Valid data files can be stored in a database that represents a source of truth and invalid data files can be routed to error handling.
1. A system for maintaining accurate data in a first database storing eligibility data associated with a plurality of subscribers, the system comprising:
one or more processors; and
one or more computer-readable media storing instructions executable by the one or more processors, wherein the instructions program the one or more processors to:
receive, from a computing device associated with a sponsor, a first data file including a first dataset of characteristics associated with the plurality of subscribers, the sponsor providing a benefit plan for the plurality of subscribers, one or more benefits of the benefit plan being specified by one or more partners;
determine first modification data associated with the first data file, the first modification data indicating a first amount of change associated with the first data file, the first amount of change indicating at least one of a number or a percentage of records in the first data file that will modify previously stored data in the first database;
retrieve, based at least in part on a characteristic of the characteristics, a first rule from a second database, the first rule being associated with a first threshold indicating a second amount of change that is determined to be permissible for the first data file;
determine whether the first amount of change associated with the first data file is permissible for the first data file based at least in part on comparing the first modification data with the first threshold;
determine that the first modification data indicates that the first amount of change is less than the first threshold; and
update the first database based at least in part on determining that the first amount of change associated with the first data file is permissible for the first data file.
2. The system as claim 1 recites, wherein the first database is a source of truth for at least the sponsor and the one or more partners.
3. The system as claim 1 recites, wherein the instructions further program the one or more processors to:
receive, from the computing device associated with the sponsor, a second data file including a third dataset of characteristics associated with the plurality of subscribers;
determine second modification data associated with the second data file, the second modification data indicating a third amount of change indicating at least one of a number or a percentage of records in the second data file that will modify the previously stored data in the first database;
retrieve the first rule from the second database;
compare the second modification data with the first threshold associated with the first rule;
determine that the second modification data indicates that the third amount of change is greater than or equal to the first threshold; and
route the second data file to an error handling component.
4. The system as claim 3 recites, wherein the instructions further program the one or more processors to process the second data file via error handling based at least in part on:
receiving, from one or more other computing devices, data indicating at least one of services provided to the plurality of subscribers or claims associated with the plurality of subscribers;
comparing the second data file with the data indicating at least one of services provided to the plurality of subscribers or claims associated with the plurality of subscribers; and
modifying the second data file based at least in part on the data indicating at least one of services provided to the plurality of subscribers or claims associated with the plurality of subscribers, wherein modifying the second data file causes the second data file to be valid.
5. The system as claim 1 recites, wherein the instructions further program the one or more processors to:
retrieve a second rule from the second database, the second rule being associated with a different characteristic of the characteristics;
compare the first modification data with a second threshold associated with the second rule, the second threshold indicating a third amount of change that is determined to be permissible for the first data file;
determine that the first modification data indicates that the first amount of change is less than the second threshold; and
update the first database based at least in part on the first amount of change being less than the first threshold and the second threshold.
6. The system as claim 5 recites, wherein the first rule and the second rule are two rules of a plurality of rules relevant to the first data file, and the instructions further program the one or more processors to determine that the first data file satisfies each rule of the plurality of rules prior to updating the first database.
7. The system as claim 1 recites, wherein the instructions further program the one or more processors to determine the first threshold based on at least one of a first attribute of the sponsor, a second attribute of a sponsor service provider, a third attribute of the partner, a time of day, a time of month, a time of year, or a season.
8. The system as claim 1 recites, wherein the instructions further program the one or more processors to:
train, via a machine learning mechanism, a data model based at least in part on previously received specified data and at least one rule;
prior to retrieving the first rule, receive specified data from the one or more partners;
determine the first rule based at least in part on the data model and the specified data;
add the first rule to the second database; and
repeatedly update the second database to include one or more additional rules based at least in part on at least one of receiving updated specified data or data associated with error handling of one or more previously analyzed data files.
9. The system as claim 8 recites, wherein the instructions further program the one or more processors to:
train, via a machine learning mechanism, a data model based at least in part on previously determined modification data associated with previously received data files; and
determine the first threshold based at least in part on the data model.
10. A computer-implemented method comprising:
receiving, by one or more server computing devices and from a computing device associated with a sponsor, a first data file including a first dataset of characteristics associated with a plurality of subscribers, the sponsor providing a benefit plan for the plurality of subscribers, one or more benefits of the benefit plan being specified by one or more partners;
determining, by the one or more server computing devices, first modification data associated with the first data file, the first modification data indicating a first amount of change associated with the first data file, the first amount of change indicating at least one of a number or a percentage of records in the first data file that will modify previously stored data in a first database storing eligibility data associated with the plurality of subscribers;
retrieving, by the one or more server computing devices and based at least in part on a characteristic of the characteristics, a first rule from a second database, the first rule being associated with a first threshold indicating a second amount of change that is determined to be permissible for the first data file;
determining, by the one or more server computing devices, whether the first amount of change associated with the data first file is permissible for the first data file based at least in part on comparing the first modification data with the first threshold;
determining, by the one or more server computing devices, that the first modification data indicates that the first amount of change is less than the first threshold; and
updating, by the one or more server computing devices, the first database based at least in part on determining that the first amount of change associated with the first data file is permissible for the first data file.
11. The computer-implemented method as claim 10 recites, wherein the first database is a source of truth for at least the sponsor and the one or more partners.
12. The computer-implemented method as claim 10 recites, further comprising:
receiving, from the computing device associated with the sponsor, a second data file including a third dataset of characteristics associated with the plurality of subscribers;
determining second modification data associated with the second data file, the second modification data indicating a third amount of change indicating at least one of a number or a percentage of records in the second data file that will modify the previously stored data in the first database;
retrieving the first rule from the second database;
comparing the second modification data with the first threshold associated with the first rule;
determining that the second modification data indicates that the third amount of change is greater than or equal to the first threshold; and
routing the second data file to an error handling component.
13. The computer-implemented method as claim 12 recites, further comprising processing the second data file via error handling based at least in part on:
receiving, from one or more other computing devices, data indicating at least one of services provided to the plurality of subscribers or claims associated with the plurality of subscribers;
comparing the second data file with the data indicating at least one of services provided to the plurality of subscribers or claims associated with the plurality of subscribers; and
modifying the second data file based at least in part on the data indicating at least one of services provided to the plurality of subscribers or claims associated with the plurality of subscribers, wherein modifying the second data file causes the second data file to be valid.
14. The computer-implemented method as claim 10 recites, further comprising:
retrieving a second rule from the second database, the second rule being associated with a different characteristic of the characteristics;
comparing the first modification data with a second threshold associated with the second rule, the second threshold indicating a third amount of change that is determined to be permissible for the first data file;
determining that the first modification data indicates that the first amount of change is less than the second threshold; and
updating the first database based at least in part on the first amount of change being less than the first threshold and the second threshold,
wherein the first rule and the second rule are two rules of a plurality of rules relevant to the first data file, and the computer-implemented method further comprises determining that the first data file satisfies each rule of the plurality of rules prior to updating the first database.
15. The computer-implemented method as claim 10 recites, further comprising determining the first threshold based on at least one of a first attribute of the sponsor, a second attribute of a sponsor service provider, a third attribute of the partner, a time of day, a time of month, a time of year, or a season.
16. The computer-implemented method as claim 10 recites, further comprising:
training, via a machine learning mechanism, a data model based at least in part on previously received specified data and at least one rule;
prior to retrieving the first rule, receiving specified data from the one or more partners;
determining the first rule based at least in part on the data model and the specified data;
adding the first rule to the second database; and
repeatedly updating the second database to include one or more additional rules based at least in part on at least one of receiving updated specified data or data associated with error handling of one or more previously analyzed data files.
17. The computer-implemented method as claim 16 recites, further comprising:
training, via a machine learning mechanism, a data model based at least in part on previously determined modification data associated with previously received data files; and
determining the first threshold based at least in part on the data model.
18. One or more non-transitory computer-readable media storing instructions executable by one or more processors, that when executed by the one or more processors, cause the one or more processors to perform operations comprising:
receiving, by one or more server computing devices and from a computing device associated with a sponsor, a data file including a dataset of characteristics associated with a plurality of subscribers, the sponsor providing a benefit plan for the plurality of subscribers, one or more benefits of the benefit plan being specified by one or more partners;
determining, by the one or more server computing devices, modification data associated with the data file, the modification data indicating a first amount of change associated with the data file, the first amount of change indicating at least one of a number or a percentage of records in the data file that will modify previously stored data in a first database storing eligibility data associated with the plurality of subscribers;
retrieving, by the one or more server computing devices and based at least in part on a characteristic of the characteristics, a rule from a second database, the rule being associated with a threshold indicating a second amount of change that is determined to be permissible for the data file;
determining, by the one or more server computing devices, whether the first amount of change associated with the data file is permissible for the data file based at least in part on comparing the modification data with the threshold;
determining, by the one or more server computing devices, that the modification data indicates that the first amount of change is less than the threshold; and
updating, by the one or more server computing devices, the first database based at least in part on determining that the first amount of change associated with the data file is permissible for the data file.
19. The one or more non-transitory computer-readable media as claim 18 recites, the operations further comprising determining the threshold utilizing a data model trained via a machine learning mechanism.