Method and apparatus for creating a lineage of a data field in a data flow system
A method and apparatus for creating a lineage for a data field is disclosed. The method may include creating a unique identifier for a data field. Also, a record of the origin of data is created when the data is inserted into the data field; however this is not required. The record of the origin of data is associated with the unique identifier for the data field. The lineage for the data field may be viewed. In so doing, the transformations to the data field may be shown.
1. A computer apparatus of creating a lineage for a data field comprising:
means for a processor;
means for creating a unique identifier for the data field;
means for creating a record of data origin when the data is inserted into the data field, the record of data origin identifying a data access query to retrieve the data from a database;
means for creating a record of transformations that are applied to the data in the data field;
means for associating the record of the origin of data and the record of transformations with the unique identifier for the data field; and
means for displaying, in response to a user on-screen selection of the data in the data field, a list of processing steps applied to the data in the data field using the unique identifier, the list of processing steps including one or more of a plurality of steps beginning with a data source step associated with the record of data origin and ending with a data viewing step.
2. The apparatus of claim 1 , further comprising:
means for displaying the data, data field and unique identifier.
3. The apparatus of claim 2 , further comprising:
means for displaying the record of origin and the record of transformations to the user.
4. A computer readable medium containing instructions which, when executed in a processing system, cause the system to:
create a unique identifier for the data field;
create a record of the data origin when the data is inserted into the data field, the record of data origin identifying a data access query to retrieve the data from a database;
create a record of transformations that are applied to the data in the data field;
associate the record of the origin of data and the record of transformations with the unique identifier for the data field; and
display, in response to a user on-screen selection of the data in the data field, a list of processing steps applied to the data in the data field using the unique identifier, the list of processing steps including one or more of a plurality of steps beginning with a data source step associated with the record of data origin and ending with a data viewing step.
5. The medium of claim 4 , wherein the executed instructions further cause the system to:
display the data, data field, and unique identifier.
6. The medium of claim 5 , wherein the executed instructions further cause the system to:
display the record of origin and the record of transformations.
7. A computer-implemented method of creating a lineage for a data field in a data processing system, comprising the steps of:
recording data origin information defining a data access query used to retrieve data from a database;
creating a record of transformations that are applied to data from the database to produce data in the data field;
defining a unique identifier for the data field associated with said data origin information and said record of transformations;
receiving a user on-screen selection of data in the data field using the unique identifier; and
in response to said user on-screen selection, displaying a list of processing steps applied to the data in the data field;
wherein the list of processing steps includes one or more steps beginning with a data source step associated with the data origin information and ending with a data viewing step.
8. The method of claim 7 wherein said data processing system incorporates a data mart and said data processing plan is a plan for processing data from said data mart.