Data asset reconstruction
According to an aspect, data asset reconstruction includes receiving a data lineage for a data asset, where the data lineage identifies a data source connected to the data asset by an intermediate process. It is determined whether the intermediate process can be used to reconstruct a lost data asset. Based on determining that the intermediate process can be used to reconstruct the lost data asset, reconstructing the lost data asset with the intermediate process and the data source.
1. A computer program product comprising:
a non-transitory computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to perform a method comprising:
receiving a first data lineage for a first lost data asset, wherein the first data lineage identifies a first data source connected to the first lost data asset by a first intermediate process, the first data source is downstream from the first lost data asset in the first data lineage, and the first intermediate process includes a process that was applied to the first lost data asset to yield the first data source;
determining whether the first intermediate process can be used to reconstruct the first lost data asset;
based on determining that the first intermediate process can be used to reconstruct the first lost data asset, including determining that the first intermediate process is reversible, reconstructing the first lost data asset by reversing the first intermediate process from the first data source to yield the first lost data asset;
identifying a second lost data asset of the first data source upon which reconstruction of the first lost data asset depends;
receiving a second data lineage for the second lost data asset, wherein the second data lineage identifies at least a second data source connected to the second lost data asset by a second intermediate process, the second data source is upstream from the second lost data asset in the second data lineage; and
based on determining that the second intermediate process is re-runnable, reconstructing the second lost data asset by re-running the second intermediate process from the second data source to yield the second lost data asset prior to reconstructing the first lost data asset,
wherein the first intermediate process is reversible if the first intermediate process is capable of being run in reverse to create the first lost data asset as output using the first data source as input, and
wherein the second intermediate process is re-runnable if the second intermediate process did not change the second data source or if the second intermediate process is a known one-to-one function.