Automated data management via machine-readable data definition files
Techniques are disclosed for automated data management. In one example, a method maintains a set of machine-readable data definition files corresponding to a set of data assets. At least one machine-readable data definition file specifies code for implementing at least one goal state associated with at least one corresponding data asset. The at least one machine-readable data definition file is executed to effectuate the at least one goal state.
1. A method, comprising:
maintaining a set of machine-readable data definition files corresponding to a set of data assets, wherein at least one machine-readable data definition file specifies code for implementing at least one goal state associated with at least one corresponding data asset; and
executing the at least one machine-readable data definition file to effectuate the at least one goal state;
wherein the at least one goal state represents one or more intended goals associated with managing the at least one corresponding data set;
wherein the step of executing includes monitoring the at least one corresponding data asset with an execution engine, the execution engine coupled to the set of machine-readable data definition files and having code configured to initiate one or more actions to enforce the at least one goal state;
wherein the at least one goal state comprises a data provisioning goal state and a data protection goal state and at least one of a data availability goal state, a data regulation goal state, a data quality goal state, a data analytics goal state, a data valuation goal state, and a capacity planning goal state;
wherein each machine-readable data definition file comprises unique identifiers corresponding to a respective given data asset of the data assets and an enterprise associated with the given data asset and at least one additional unique identifier corresponding to at least one of a state of the given data asset and a location of the given data asset; and
wherein the maintaining and executing steps are implemented via at least one processing device comprising a processor and a memory.
2. The method of claim 1 , further comprising maintaining a mapping between the set of machine-readable data definition files and the set of data assets.
3. The method of claim 1 , further comprising editing of the set of machine-readable data definition files.
4. The method of claim 3 , wherein the step of editing the set of machine-readable data definition files further comprises, upon the addition of one or more new data assets to the set of data assets, creating one or more new machine-readable data definition files and adding the one or more new machine-readable data definition files to the set of machine-readable data definition files.
5. The method of claim 3 , wherein the step of editing the set of machine-readable data definition files further comprises, upon the deletion of one or more existing data assets from the set of data assets, deleting one or more existing machine-readable data definition files in the set of machine-readable data definition files.
6. The method of claim 3 , wherein the step of editing the set of machine-readable data definition files further comprises, upon the modification of one or more existing data assets in the set of data assets, modifying one or more existing machine-readable data definition files in the set of machine-readable data definition files.
7. The method of claim 3 , wherein the step of editing the set of machine-readable data definition files further comprises validating one or more edits to the set of machine-readable data definition files before the one or more edits are implemented.
8. The method of claim 1 , further comprising the step of one or more new data assets added to the set of data assets inheriting one or more existing machine-readable data definition files.
9. The method of claim 1 , further comprising the step of one or more new data assets added to the set of data assets assuming one or more existing machine-readable data definition files.
10. The method of claim 1 , wherein the step of maintaining the set of machine-readable data definition files further comprises maintaining version control data for the set of machine-readable data definition files.
11. The method of claim 1 , wherein the step of executing the at least one machine-readable data definition file to effectuate the at least one goal state further comprises utilizing one or more application programming interfaces to effectuate the at least one goal state upon execution of the at least one machine-readable data definition file, the one or more application programming interfaces integrated with the execution engine and actuated in response to a request from the execution engine.
12. The method of claim 1 , wherein the code for at least one machine-readable data definition file further specifies code for enabling infrastructure for one of storing and processing at least one corresponding data asset.
13. An article of manufacture comprising a non-transitory processor-readable storage medium having encoded therein executable code of one or more software programs, wherein the one or more software programs when executed by one or more processing devices implement the steps of:
maintaining a set of machine-readable data definition files corresponding to a set of data assets, wherein at least one machine-readable data definition file specifies code for implementing at least one goal state associated with at least one corresponding data asset; and
executing the at least one machine-readable data definition file to effectuate the at least one goal state;
wherein the at least one goal state represents one or more intended goals associated with managing the at least one corresponding data set;
wherein the step of executing includes monitoring the at least one corresponding data asset with an execution engine, the execution engine coupled to the set of machine-readable data definition files and having code configured to initiate one or more actions to enforce the at least one goal state;
wherein the at least one goal state comprises a data provisioning goal state and a data protection goal state and at least one of a data availability goal state, a data regulation goal state, a data quality goal state, a data analytics goal state, a data valuation goal state, and a capacity planning goal state; and
wherein each machine-readable data definition file comprises unique identifiers corresponding to a respective given data asset of the data assets and an enterprise associated with the given data asset and at least one additional unique identifier corresponding to at least one of a state of the given data asset and a location of the given data asset.
14. A system comprising:
one or more processors operatively coupled to one or more memories configured to:
maintain a set of machine-readable data definition files corresponding to a set of data assets, wherein at least one machine-readable data definition file specifies code for implementing at least one goal state associated with at least one corresponding data asset; and
execute the at least one machine-readable data definition file to effectuate the at least one goal state;
wherein the at least one goal state represents one or more intended goals associated with managing the at least one corresponding data set;
wherein executing includes monitoring the at least one corresponding data asset with an execution engine, the execution engine coupled to the set of machine-readable data definition files and having code configured to initiate one or more actions to enforce the at least one goal state;
wherein the at least one goal state comprises a data provisioning goal state and a data protection goal state and at least one of a data availability goal state, a data regulation goal state, a data quality goal state, a data analytics goal state, a data valuation goal state, and a capacity planning goal state; and
wherein each machine-readable data definition file comprises unique identifiers corresponding to a respective given data asset of the data assets and an enterprise associated with the given data asset and at least one additional unique identifier corresponding to at least one of a state of the given data asset and a location of the given data asset.
15. The system of claim 14 , wherein the one or more processors are further configured to maintain a mapping between the set of machine-readable data definition files and the set of data assets.
16. The system of claim 14 , wherein the one or more processors are further configured to edit the set of machine-readable data definition files.
17. The system of claim 14 , wherein the one or more processors are further configured to maintain version control data for the set of machine-readable data definition files.
18. The system of claim 14 , wherein the one or more processors are further configured to utilize one or more application programming interfaces to effectuate the at least one goal state upon execution of the at least one machine-readable data definition file, the one or more application programming interfaces integrated with the execution engine and actuated in response to a request from the execution engine.
19. The article of manufacture of claim 13 wherein editing the set of machine-readable data definition files further comprises at least one of the following:
upon the addition of one or more new data assets to the set of data assets, creating one or more new machine-readable data definition files and adding the one or more new machine-readable data definition files to the set of machine-readable data definition files;
upon the deletion of one or more existing data assets from the set of data assets, deleting one or more existing machine-readable data definition files in the set of machine-readable data definition files; and
upon the modification of one or more existing data assets in the set of data assets, modifying one or more existing machine-readable data definition files in the set of machine-readable data definition files.
20. The system of claim 14 wherein editing the set of machine-readable data definition files further comprises at least one of the following:
upon the addition of one or more new data assets to the set of data assets, creating one or more new machine-readable data definition files and adding the one or more new machine-readable data definition files to the set of machine-readable data definition files;
upon the deletion of one or more existing data assets from the set of data assets, deleting one or more existing machine-readable data definition files in the set of machine-readable data definition files; and
upon the modification of one or more existing data assets in the set of data assets, modifying one or more existing machine-readable data definition files in the set of machine-readable data definition files.