Systems and methods for determining relationships among data elements

1. A data processing system, comprising:

at least one computer hardware processor; and

at least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by the at least one computer hardware processor, cause the at least one computer hardware processor to perform:

obtaining a first data lineage representing relationships among a plurality of physical data elements, the first data lineage being generated at least in part by performing at least one of: (a) analyzing source code of at least one computer program configured to access at least some of the plurality of physical data elements; and (b) analyzing information obtained during runtime of the at least one computer program;

obtaining, based at least in part on user input, a second data lineage representing relationships among a plurality of business data elements;

obtaining an association between at least some of the plurality of physical data elements of the first data lineage and at least some of the plurality of business data elements of the second data lineage; and

generating, based on the association between the plurality of physical data elements and the plurality of business data elements, an indication of agreement or discrepancy between the first data lineage and the second data lineage.

2. The data processing system of claim 1 , wherein generating the indication of agreement or discrepancy comprises:

displaying a visualization of the second data lineage showing the indication of agreement or discrepancy.

3. The data processing system of claim 2 , wherein the second data lineage comprises a first link representing a first dependency between two business data elements, and wherein displaying the visualization of the second data lineage comprises displaying the link in one manner when there is a dependency in the first data lineage corresponding to the first dependency and in another manner when there is not a dependency in the first data lineage corresponding to the first dependency.

4. The data processing system of claim 2 , wherein obtaining the first data lineage comprises analyzing the source code of the at least one computer program.

5. The data processing system of claim 2 , wherein obtaining the first data lineage comprises analyzing the information obtained during runtime of the at least one computer program.

6. The data processing system of claim 1 , wherein generating the indication of agreement or discrepancy comprises:

determining, based on the association between the plurality of physical data elements and the plurality of business data elements, whether there is one or more discrepancies among the first data lineage, the second data lineage, and the obtained association.

7. The data processing system of claim 6 ,

wherein the plurality of physical data elements comprises a first physical data element,

wherein the plurality of business data elements comprises a first business data element,

wherein the association indicates that the first physical data element and the first business data element are associated, and

wherein the determining comprises determining that a first set of one or more sources of data identified in the first data lineage as being used to obtain the first physical data element is different from a second set of one or more sources of data identified in the second data lineage as being used to obtain the first business data element.

8. The data processing system of claim 6 , wherein acts of obtaining the first data lineage and determining whether there is a discrepancy are performed repeatedly according to a specified schedule.

9. The data processing system of claim 1 , wherein obtaining the first data lineage comprises generating the first data lineage at least in part by performing at least one of analyzing the source code of the at least one computer program and analyzing the information obtained during runtime of the at least one computer program.

10. The data processing system of claim 1 , wherein the at least one computer program comprises a computer program implemented as a dataflow graph.

11. The data processing system of claim 1 , wherein obtaining the association between the at least some of the plurality of physical data elements of the first data lineage and the at least some of the plurality of business data elements of the second data lineage comprises generating the association based on user input provided via a graphical user interface.

12. The data processing system of claim 1 , wherein the association comprises an association between a first physical data element of the plurality of physical data elements and a first business data element of the plurality of business data elements, and wherein the at least one computer hardware processor is further configured to perform:

determining, based at least in part on the association between the first physical data element and the first business data element, a measure of data quality for the first business data element.

13. The data processing system of claim 12 , wherein determining the measure of data quality for the first business data element comprises:

performing an analysis of data quality of data in the first physical data element based at least in part on one or more data quality rules associated with the data in the first physical data element.

14. The data processing system of claim 12 , wherein the measure of data quality for the first business element includes a measure of one or more of accuracy, completeness, and validity.

15. A method, comprising:

using at least one computer hardware processor to perform:

obtaining a first data lineage representing relationships among a plurality of physical data elements, the first data lineage being generated at least in part by performing at least one of: (a) analyzing source code of at least one computer program configured to access at least some of the plurality of physical data elements; and (b) analyzing information obtained during runtime of the at least one computer program;

obtaining, based at least in part on user input, a second data lineage representing relationships among a plurality of business data elements;

obtaining an association between at least some of the plurality of physical data elements of the first data lineage and at least some of the plurality of business data elements of the second data lineage; and

generating, based on the association between the plurality of physical data elements and the plurality of business data elements, an indication of agreement or discrepancy between the first data lineage and the second data lineage.

16. A data processing system for determining a measure of data quality for one or more business data elements, the system comprising:

at least one computer hardware processor; and

at least one non-transitory computer-readable storage medium storing processor executable instructions that, when executed by the at least one computer hardware processor, cause the at least one computer hardware processor to perform:

obtaining a first data lineage representing relationships among a plurality of physical data elements, the first data lineage being generated at least in part by performing at least one of analyzing source code of at least one computer program configured to access at least some of the plurality of physical data elements and analyzing information obtained during runtime of the at least one computer program;

obtaining, based at least in part on user input, a second data lineage representing relationships among a plurality of business data elements;

obtaining an association between at least some of the plurality of physical data elements of the first data lineage and at least some of the plurality of business data elements of the second data lineage, the association including an association between a first physical data element of the plurality of physical data elements and a first business data element of the plurality of business data elements; and

determining a measure of data quality for the first business data element based at least in part on at least one data quality measure associated with the first physical data element and the association between the first physical data element and the first business data element.

17. The data processing system of claim 16 , wherein determining the measure of data quality for the first business data element comprises:

performing an analysis of data quality of data in the first physical data element based at least in part on one or more data quality rules associated with the data in the first physical data element to obtain the at least one data quality measure associated with the first physical data element.

18. The data processing system of claim 16 , wherein the measure of data quality for the first business element includes a measure of one or more of accuracy, completeness, and validity.

19. The data processing system of claim 16 , wherein obtaining the first data lineage comprises receiving the first data lineage after it has been generated.

20. The data processing system of claim 16 , wherein obtaining the first data lineage comprises generating the first data lineage.

21. The data processing system of claim 20 , wherein generating the first data lineage comprises analyzing the source code of the at least one computer program.

22. The data processing system of claim 20 , wherein generating the first data lineage comprises and analyzing the information obtained during runtime of the at least one computer program.

23. The data processing system of claim 16 , wherein the at least one computer program comprises a computer program implemented as a dataflow graph.

24. The data processing system of claim 16 , wherein obtaining the association between the at least some of the plurality of physical data elements of the first data lineage and the at least some of the plurality of business data elements of the second data lineage comprises generating the association between the first physical data element and the first business data element is performed based on user input associating the first physical data element and the first business data element.

25. The data processing system of claim 16 , further comprising:

determining, based on the association between the plurality of physical data elements and the plurality of business data elements, whether there is a discrepancy between the first data lineage and the second data lineage.

26. The data processing system of claim 16 , wherein acts of obtaining the first data lineage and determining the measure of data quality are performed repeatedly according to a specified schedule.