File record validation
Techniques for validating and modifying file records are disclosed. A reference specification is obtained, indicating valid data labels and data points, where each data point is classified into a data label. A semantic assessment is performed between a file record and the reference specification to determine semantically conforming data descriptors and data labels. It is then determined whether aspects of data elements associated with the data descriptors conform with correlated aspects of data points linked to the data labels. Based on the determination, a data modification workflow is triggered to transform non-conforming aspects of data elements. The file record is subsequently modified to include the transformed data elements. The present subject matter may enable automated validation and modification of file records to ensure conformity with the reference specification.
1 . A system comprising:
a processor to:
receive a file record for validation, the file record comprising a plurality of data descriptors, each having one or more data elements associated therewith, wherein each of the plurality of data descriptors indicates an attribute of one or more data records associated therewith;
obtain, in response to receiving a file record for validation thereof, a reference specification indicating a plurality of valid data labels and a plurality of valid data points, wherein each of the plurality of data points is classified into a data label from amongst the plurality of data labels;
parse the plurality of data descriptors and the plurality of data labels to determine respective semantic representations of the plurality of data descriptors and the plurality of data labels;
perform a semantic assessment between the file record and the reference specification to determine one or more data descriptors, from amongst the plurality of data descriptors, semantically conforming with one data label, from amongst the plurality of data labels of the reference specification, wherein the semantic assessment is performed to map the one or more data descriptors to the plurality of data points, and wherein the semantic assessment comprises:
computing a similarity score between the semantic representations of the plurality of data descriptors and the plurality of data labels; and
identifying one or more descriptor-label pairs based on comparison of the similarity score to a threshold similarity score;
determine, for each semantically conforming the one or more descriptor-label pair, whether an aspect of one or more data elements, associated with that data descriptor, conforms with a correlated aspect associated with each of the plurality of data points linked with the one data label, wherein the aspect comprises at least a format of the one or more data elements;
trigger, based on the determination, a data modification workflow to cause modification of the aspect of the one or more data elements, wherein the modification is to transform the aspect of the one or more data elements based on the correlated aspect; and
modify, in response to triggering of the data modification workflow, the file record to include the one or more data elements having the transformed aspect.
2 . The system of claim 1 , wherein the aspect of one or more data elements is a format of the one or more data elements.
3 . The system of claim 1 , wherein the processor is to generate a data representation signal to cause rendering of the modified file record with the one or more data elements having the transformed aspect.
4 . The system of claim 3 , wherein the processor is to cause rendering of at least one modification indicator in the modified file record to distinguishingly indicate the one or more data elements, having the transformed aspect, from other of the one or more data elements.
5 . The system of claim 1 , wherein, prior to performance of the semantic assessment, the processor is to perform a preliminary data arrangement workflow to:
determine a semantic similarity between each data element in the file record and each of the plurality of data descriptors in the file record;
identify, based on the semantic similarity, one or more data elements semantically conforming with a data descriptor, different from the data descriptor already linked with the one or more data elements, from amongst the plurality of data descriptors; and
associate, based on the identification, the one or more data elements with the semantically conforming data descriptor.
6 . The system of claim 1 , wherein the processor is to:
obtain, in response to receiving the file record, a set of acceptable data elements;
compare each data element in the file record with the set of acceptable data elements to identify one or more data elements anomalous from the set of acceptable data; and
generate, based on the comparison, an alert generation signal to cause rendering of at least one alert to indicate presence of the anomalous one or more data elements in the file record.
7 . The system of claim 6 , wherein the processor is to replace the anomalous one or more data elements with a default data element.
8 . The system of claim 1 , wherein the processor is to:
determine, based on the semantic representation of each of the plurality of data labels and the semantic representation of each of the plurality of data descriptors, the one or more data descriptors semantically conforming with the one data label.
9 . The system of claim 1 , wherein the processor is to:
compute the similarity score for quantifying a semantic conformity between the semantic representation of each of the plurality of data labels and the semantic representation of each of the plurality of data descriptors; and
compare the similarity score with the threshold similarity score to identify the one or more data descriptors having semantic conformity with at least one data label from amongst the plurality of data labels.
10 . The system of claim 1 , wherein each of the plurality of data labels indicates a characteristic of the one or more data points linked therewith, and wherein each of the plurality of data descriptors indicates at least one attribute of the one or more data elements associated therewith.
11 . A method comprising:
receiving a file record for validation, the file record comprising a plurality of data descriptors, each having one or more data elements associated therewith, wherein each of the plurality of data descriptors indicates an attribute of the one or more data records associated therewith;
obtaining, in response to receiving the file record, a reference specification derived based on a configuration file, the reference specification indicating a plurality of valid data labels and a plurality of valid data points, wherein each of the plurality of data points is classified into a data label from amongst the plurality of data labels;
parsing the plurality of data descriptors and the plurality of data labels to determine respective semantic representations of the plurality of data descriptors and the plurality of data labels;
performing a semantic assessment for each of the plurality of data labels and each of the plurality of data descriptors to determine one or more data descriptors, from amongst the plurality of data descriptors, semantically matching with a data label, from amongst the plurality data labels of the reference specification, wherein the semantic assessment is performed to map the one or more data descriptors to the plurality of data points, and wherein the semantic assessment comprises of:
computing a similarity score between the semantic representations of the plurality of data descriptors and the plurality of data labels; and
identifying one or more descriptor-label pairs based on comparison of the similarity score to a threshold similarity score;
determining, for each semantically matching the one or more descriptor-label pair, whether an aspect of one or more data elements, associated with that data descriptor, conforms with a correlated aspect associated with each of the plurality of data points linked with the data label, wherein the aspect comprises at least a format of the one or more data elements;
transforming, based on the determining, the aspect of the one or more data elements based on the correlated aspect; and
generating a data representation signal to cause rendering of a modified file record with the one or more data elements having the transformed aspect.
12 . The method of claim 11 , wherein the method further comprises parsing the configuration file to derive the reference specification and one or more aspects associated with each of the plurality of data labels and the data elements.
13 . The method of claim 11 , wherein the method further comprises causing rendering of at least one modification indicator in the modified file record to distinguishingly indicate the one or more data elements, having the transformed aspect, from other of the one or more data elements.
14 . The method of claim 11 , wherein the method further comprises:
determining a semantic similarity between each data element in the file record and each of the plurality of data descriptors in the file record;
identifying, based on the semantic similarity, one or more data elements semantically matching with a data descriptor, different from the data descriptor linked with the one or more data elements, from amongst the plurality of data descriptors; and
associating, based on the identifying, the one or more data elements with the semantically matching data descriptor.
15 . The method of claim 11 , wherein the method further comprises:
comparing the semantic representation of each of the plurality of data labels and the semantic representation of each of the plurality of data descriptors to determine the one or more data descriptors semantically matching with the data label.
16 . The method of claim 11 , the method further comprising causing rendering of a feedback option to receive at least one of a positive feedback and a negative feedback, the positive feedback indicating acceptance of the one or more data elements having the transformed aspect and the negative feedback indicating rejection of the one or more data elements having the transformed aspect.
17 . A non-transitory computer-readable medium comprising instructions, the instructions being executable by a processing resource to:
receive a file record for validation, the file record comprising a plurality of data descriptors, each having one or more data elements associated therewith, wherein each of the plurality of data descriptors indicates an attribute of the one or more data records associated therewith;
obtain, in response to receiving a file record for validation thereof, a reference specification indicating a plurality of valid data labels and a plurality of valid data points, wherein each of the plurality of data points is classified into a data label from amongst the plurality of data labels;
parse the plurality of data descriptors and the plurality of data labels to determine respective semantic representations of the plurality of data descriptors and the plurality of data labels;
perform a semantic assessment for each of the plurality of data labels and each of the plurality of data descriptors to determine a semantic score for each of the plurality of data descriptors of the file record, the semantic score indicating a measure of semantic similarity between each of the plurality of data descriptors and each of the plurality of data labels, wherein the semantic assessment is performed to map the one or more data descriptors to the plurality of data points, and wherein the semantic assessment comprises of:
computing a similarity score between the semantic representations of the plurality of data descriptors and the plurality of data labels; and
identifying one or more descriptor-label pairs based on comparison of the similarity score to a threshold similarity score;
identify, based on the semantic score of each of the plurality of data descriptors, one or more data descriptors, from amongst the plurality of data descriptors, semantically matching with a data label, from amongst the plurality data labels of the reference specification;
determine whether an aspect of one or more data elements, associated with each of the one or more data descriptors semantically matching with the data label, conforms with a correlated aspect associated with each of the plurality of data points linked with the data label, wherein the aspect comprises at least a format of the one or more data elements;
trigger, based on the determination, transformation of the aspect of the one or more data elements based on the correlated aspect, wherein the transformation is to align the aspect of the one or more data elements with the correlated aspect;
modify, in response to triggering of the transformation, the file record to include the one or more data elements having the transformed aspect; and
cause rendering of the modified file record with the one or more data elements having the transformed aspect.
18 . The non-transitory computer-readable medium of claim 17 , the instructions being executable by the processing resource to cause rendering of at least one modification indicator in the modified file record to distinguishingly indicate the one or more data elements, having the transformed aspect, from other of the one or more data elements.
19 . The non-transitory computer-readable medium of claim 17 , the instructions being executable by the processing resource to:
determine a semantic similarity between each data element in the file record and each of the plurality of data descriptors in the file record;
identify, based on the semantic similarity, one or more data elements semantically conforming with a data descriptor, different from the data descriptor already linked with the one or more data elements, from amongst the plurality of data descriptors; and
associate, based on the identification, the one or more data elements with the semantically conforming data descriptor.
20 . The non-transitory computer-readable medium of claim 17 , the instructions being executable by the processing resource to cause rendering of a feedback option to receive at least one of a positive feedback and a negative feedback, the positive feedback indicating acceptance of the one or more data elements having the transformed aspect and the negative feedback indicating rejection of the one or more data elements having the transformed aspect.