Data reconciliation and inconsistency determination for posted entries
A system and a method are disclosed for receiving an entry, the entry comprising first content and a metadata tag corresponding to a classification, the first content populated by a first source. A rules engine determines that the first content comprises a data field associated with at least one of a plurality of reconciliation policies. Responsive to determining that the first content comprises a data field associated with at least one of the plurality of reconciliation policies, the rules engine selects a reconciliation policy based on the metadata tag. The rules engine retrieves, from a second source, second content associated with the data field, inputs the first content and the second content into a model, the model selected based on the reconciliation policy, the model generating an output, and performs a remediation action based on the output.
1. A non-transitory computer-readable medium comprising memory with instructions encoded thereon, the instructions, when executed, causing one or more processors to perform operations, the instructions comprising instructions to:
receive an entry, the entry comprising a payload having first content, and a metadata tag separate from the payload corresponding to a classification of the first content and describing at least one attribute of the first content, the first content populated by a first source;
search the payload for a data field within the first content that is associated with at least one of a plurality of reconciliation policies;
determine that the first content comprises a data field associated with at least one of the plurality of reconciliation policies;
responsive to determining that the first content comprises a data field associated with at least one of the plurality of reconciliation policies:
select a reconciliation policy based on the metadata tag; and
validate the first content against second content from a second source based on the selected reconciliation policy.
2. The non-transitory computer-readable medium of claim 1 , wherein the instructions to validate the first content against the second content from the second source based on the selected reconciliation policy comprise instructions to perform a remediation action associated with the selected reconciliation policy.
3. The non-transitory computer-readable medium of claim 2 , wherein the remediation action comprises:
determining, based on the reconciliation policy, a destination for an alert; and
routing the alert to the destination.
4. The non-transitory computer-readable medium of claim 1 , wherein the instructions further comprise instructions to:
determine a result of the validation matches an alert condition; and
transmit an alert to a user responsive to determining that the result of the validation matches the alert condition.
5. The non-transitory computer-readable medium of claim 4 , wherein the instructions further comprise instructions to, responsive to determining that the result of the validation does not match the alert condition, store data representative of the result of the validation to memory.
6. The non-transitory computer-readable medium of claim 5 , wherein the data representative of the result of the validation is indexed and searchable.
7. The non-transitory computer-readable medium of claim 1 , wherein the metadata tag is applied to the entry based on the classification of contents of the payload.
8. A method comprising:
receiving, by a data management system, an entry, the entry comprising a payload having first content, and a metadata tag separate from the payload corresponding to a classification of the first content and describing at least one attribute of the first content, the first content populated by a first source;
searching, by the data management system, the payload for a data field within the first content that is associated with at least one of a plurality of reconciliation policies;
determining, by the data management system, that the first content comprises a data field associated with at least one of the plurality of reconciliation policies;
responsive to determining that the first content comprises a data field associated with at least one of the plurality of reconciliation policies:
selecting, by the data management system, a reconciliation policy based on the metadata tag; and
validating, by the data management system, the first content against second content from a second source based on the selected reconciliation policy.
9. The method of claim 8 , wherein validating the first content against the second content from the second source based on the selected reconciliation policy comprises performing a remediation action associated with the selected reconciliation policy.
10. The method of claim 9 , wherein the remediation action comprises:
determining, based on the reconciliation policy, a destination for an alert; and
routing the alert to the destination.
11. The method of claim 8 , further comprising:
determining a result of the validation matches an alert condition; and
transmitting an alert to a user responsive to determining that the result of the validation matches the alert condition.
12. The method of claim 11 , further comprising, responsive to determining that the result of the validation does not match the alert condition, storing data representative of the result of the validation to memory.
13. The method of claim 12 , wherein the data representative of the result of the validation is indexed and searchable.
14. The method of claim 8 , wherein the metadata tag is applied to the entry based on the classification of contents of the payload.
15. A system comprising:
memory with instructions encoded thereon; and
one or more processors that, when executing the instructions, are caused to perform operations comprising:
receiving an entry, the entry comprising a payload having first content, and a metadata tag separate from the payload corresponding to a classification of the first content and describing at least one attribute of the first content, the first content populated by a first source;
searching the payload for a data field within the first content that is associated with at least one of a plurality of reconciliation policies;
determining that the first content comprises a data field associated with at least one of the plurality of reconciliation policies;
responsive to determining that the first content comprises a data field associated with at least one of the plurality of reconciliation policies:
selecting a reconciliation policy based on the metadata tag; and
validating the first content against second content from a second source based on the selected reconciliation policy.
16. The system of claim 15 , wherein validating the first content against the second content from the second source based on the selected reconciliation policy comprises performing a remediation action associated with the selected reconciliation policy.
17. The system of claim 16 , wherein the remediation action comprises:
determining, based on the reconciliation policy, a destination for an alert; and
routing the alert to the destination.
18. The system of claim 15 , the operations further comprising:
determining a result of the validation matches an alert condition; and
transmitting an alert to a user responsive to determining that the result of the validation matches the alert condition.
19. The system of claim 18 , the operations further comprising, responsive to determining that the result of the validation does not match the alert condition, storing data representative of the result of the validation to memory.
20. The system of claim 19 , wherein the data representative of the result of the validation is indexed and searchable.