Synchronizing masking jobs between different masking engines in a data processing system
A data processing system maintaining a plurality of masking engines is configured to synchronize masking jobs between different masking engines, in order to ensure consistent outputs between the different engines for the same job. Upon identifying a masking job object maintained by a first masking engine to be exported, dependencies of the job object are traversed to identify all objects depended on by the job object, which are used to form an export payload. At least object of the payload is a reference object that references other objects maintained by the first masking engine. When importing the export payload onto a second masking engine, the export payload is analyzed to identify revision information of the referenced object. The identified revision information is compared with revision information of a corresponding object maintained by the second masking engine, and the referenced objects is imported based upon a result of the comparison.
1. A method for synchronizing masking job objects between masking engines, comprising:
at a first masking engine:
identifying a first object maintained by the first masking engine to be exported, the first job corresponding to a masking job configured to replace a subset of data within a corpus of data identified as sensitive with masked data;
traversing one or more dependencies of the first object to identify a set of objects that the first object is dependent upon, the set of objects indicating how the masking job is to be performed and including at least one rule set object indicating a structure of data to be processed by the masking job corresponding to the first object, and wherein at least one object of the set of objects is a reference object referencing a second object maintained by the first masking engine and having associated revision information; and
generating an export payload containing the first object and the set of identified objects;
at a second masking engine:
receiving the export payload generated at the first masking engine;
identifying the revision information of the reference object contained within the export payload;
comparing the identified revision information with revision information of a corresponding object maintained by the second masking engine;
importing the reference objects of the export payload based upon a result of the comparison; and
performing the masking job corresponding to the export payload at the second masking engine in accordance with the objects of the export payload.
2. The method of claim 1 , wherein the second object referenced by the reference object corresponds to a set of instructions for mapping one or more input data values to masked data values.
3. The method of claim 1 , further comprising:
responsive to a determination that the identified revision information matches the revision information of the corresponding object, not importing the reference object to the second masking engine.
4. The method of claim 3 , further comprising:
importing the first object onto the second masking engine, and configuring the first object to depend upon the corresponding object.
5. The method of claim 1 , wherein the revision information of the reference object comprises an object identifier identifying the reference object, and a revision identifier identifying a revision state of the reference object.
6. The method of claim 5 , further comprising:
responsive to a determination that the object identifier of the identified revision information does not match that of the revision information of the corresponding object, importing the second object into the second masking engine, regardless of whether the revision identifier matches that of the corresponding object.
7. The method of claim 1 , wherein the first set of objects further includes a file format object indicating a format of the data to be processed.
8. The method of claim 1 , wherein the first set of objects further includes a target connector object comprising information for accessing a database storing at least a portion of the data.
9. The method of claim 1 , further comprising generating a revision hash for the reference object corresponding to an aggregation of the revision information of the reference object and at least one other object of the set of objects that the reference object is dependent on, and wherein comparing the identified revision information with revision information of a corresponding object maintained by the second masking engine comprises comparing the generated revision hash with a revision hash of the corresponding object.
10. A computer readable non-transitory storage medium, storing instructions for:
at a first masking engine:
identifying a first object maintained by the first masking engine to be exported, the first job corresponding to a masking job configured to replace a subset of data within a corpus of data identified as sensitive with masked data;
traversing one or more dependencies of the first object to identify a set of objects that the first object is dependent upon, the set of objects indicating how the masking job is to be performed and including at least one rule set object indicating a structure of data to be processed by the masking job corresponding to the first object, and wherein at least one object of the set of objects is a reference object referencing a second object maintained by the first masking engine and having associated revision information; and
generating an export payload containing the first object and the set of identified objects;
at a second masking engine:
receiving the export payload generated at the first masking engine;
identifying the revision information of the reference object contained within the export payload;
comparing the identified revision information with revision information of a corresponding object maintained by the second masking engine;
importing the reference objects of the export payload based upon a result of the comparison; and
performing the masking job corresponding to the export payload at the second masking engine in accordance with the objects of the export payload.
11. The computer readable non-transitory storage medium of claim 10 , wherein the second object referenced by the reference object corresponds to a set of instructions for mapping one or more input data values to masked data values.
12. The computer readable non-transitory storage medium of claim 10 , further storing instructions for:
responsive to a determination that the identified revision information matches the revision information of the corresponding object, not importing the reference object to the second masking engine.
13. The computer readable non-transitory storage medium of claim 12 , further storing instructions for:
importing the first object onto the second masking engine, and configuring the first object to depend upon the corresponding object.
14. The computer readable non-transitory storage medium of claim 10 , wherein the revision information of the reference object comprises an object identifier identifying the reference object, and a revision identifier identifying a revision state of the reference object.
15. The computer readable non-transitory storage medium of claim 14 , further storing instructions for:
responsive to a determination that the object identifier of the identified revision information does not match that of the revision information of the corresponding object, importing the second object into the second masking engine, regardless of whether the revision identifier matches that of the corresponding object.
16. The computer readable non-transitory storage medium of claim 10 , wherein the first set of objects further includes a file format object indicating a format of the data to be processed.
17. The computer readable non-transitory storage medium of claim 10 , wherein the first set of objects further includes a target connector object comprising information for accessing a database storing at least a portion of the data.
18. The computer readable non-transitory storage medium of claim 10 , further storing instructions for generating a revision hash for the reference object corresponding to an aggregation of the revision information of the reference object and at least one other object of the set of objects that the reference object is dependent on, and wherein comparing the identified revision information with revision information of a corresponding object maintained by the second masking engine comprises comparing the generated revision hash with a revision hash of the corresponding object.