System and method for interactive discovery of inter-data set relationships
Provided are methods and systems comprising determining one or more relationships between a plurality of data sets, determining a score for each of the one or more relationships, generating a graphical data set object for each of the plurality of data sets, classifying each graphical data set object as connected or unconnected based on the score for each of the one or more relationships, generating a graphical connector object between connected graphical data set objects, and outputting the connected graphical data set objects with corresponding graphical connector objects and the unconnected graphical data set objects.
1. A method comprising:
generating, for a first data set, a first graphical data set object comprising a graphical size that is based on a number of rows of one or more data tables within the first data set;
generating, for a second data set, a second graphical data set object comprising a graphical size that is based on a number of rows of one or more data tables within the second data set;
outputting, at a graphical user interface (GUI), the first graphical data set object and the second graphical data set object;
determining, based on a first user interaction, a score for one or more relationships between the first data set and the second data set, wherein the first user interaction comprises movement, via the GUI, of the first graphical data set object closer in position within the GUI to the second graphical data set object;
generating, based on the first user interaction, and based on the score for the one or more relationships satisfying a threshold, a graphical connector object that connects the first graphical data set object to the second graphical data set object within the GUI to indicate the score for the one or more relationships satisfies the threshold; and
outputting, via the GUI, the first graphical data set object and the second graphical data set object with the graphical connector object.
2. The method of claim 1 , wherein the first data set and the second data set comprise one or more common attributes.
3. The method of claim 1 , wherein the first data set and the second data set comprise one or more common values.
4. The method of claim 1 , wherein determining the score for the one or more relationships between the first data set and the second data set is based on a profiling/classification parameter that comprises one or more of:
one or more ratios of unique values in one or more fields of data within the one or more data tables associated with each of the first data set and the second data set;
one or more ratios of a size of an overlapping set of values from the one or more fields;
one or more ratios of the size of the overlapping set of values from the one or more fields;
a history of user decisions made against the same or similar data;
metadata supplied by an external system, a developer, or an administrator;
a hint supplied by the developer or the administrator; or
a classification score that identifies a potential lookup table.
5. The method of claim 1 , wherein the graphical size of each of the first graphical data set object and the second graphical data set object is based on logarithmic scaling applied to the number of rows of the one or more data tables within each data set.
6. The method of claim 1 , further comprising outputting a data model based on the first data set and the second data set.
7. The method of claim 1 , further comprising:
receiving, via the GUI, a selection of the first graphical data set object; and
outputting, based on the selection, one or more of:
a preview of the first data set; or
a preview of a relationship of the one or more relationships.
8. An apparatus comprising:
one or more processors; and
memory storing processor-executable instructions that, when executed by the one or more processors, cause the apparatus to:
generate, for a first data set, a first graphical data set object comprising a graphical size that is based on a number of rows of one or more data tables within the first data set;
generate, for a second data set, a second graphical data set object comprising a graphical size that is based on a number of rows of one or more data tables within the second data set;
output, at a graphical user interface (GUI), the first graphical data set object and the second graphical data set object;
determine, based on a first user interaction, a score for one or more relationships between the first data set and the second data set, wherein the first user interaction comprises movement, via the GUI, of the first graphical data set object closer in position within the GUI to the second graphical data set object;
generate, based on the first user interaction, and based on the score for the one or more relationships satisfying a threshold, a graphical connector object that connects the first graphical data set object to the second graphical data set object within the GUI to indicate the score for the one or more relationships satisfies the threshold; and
output, via the GUI, the first graphical data set object and the second graphical data set object with the graphical connector object.
9. The apparatus of claim 8 , wherein the first data set and the second data set comprise one or more common attributes.
10. The apparatus of claim 8 , wherein the first data set and the second data set comprise one or more common values.
11. The apparatus of claim 8 , wherein the processor-executable instructions that cause the apparatus to determine the score for the one or more relationships between the first data set and the second data set further cause the apparatus to determine the score for the one or more relationships between the first data set and the second data set based on a profiling/classification parameter that comprises one or more of:
one or more ratios of unique values in one or more fields of data within the one or more data tables associated with each of the first data set and the second data set; one or more ratios of a size of an overlapping set of values from the one or more fields
one or more ratios of the size of the overlapping set of values from the one or more fields;
a history of user decisions made against the same or similar data;
metadata supplied by an external system, a developer, or an administrator;
a hint supplied by the developer or the administrator; or
a classification score that identifies a potential lookup table.
12. The apparatus of claim 8 , wherein the graphical size of each of the first graphical data set object and the second graphical data set object is based on logarithmic scaling applied to the number of rows of the one or more data tables within each data set.
13. The apparatus of claim 8 , wherein the processor-executable instructions further cause the apparatus to output a data model based on the first data set and the second data set.
14. The apparatus of claim 8 , wherein the processor-executable instructions further cause the apparatus to:
receive, via the GUI, a selection of the first graphical data set object; and
output, based on the selection, one or more of:
a preview of the first data set; or
a preview of a relationship of the one or more relationships.
15. A non-transitory computer-readable medium comprising processor-executable instructions that, when executed by one or more processors, cause the one or more processors to:
generate, for a first data set, a first graphical data set object comprising a graphical size that is based on a number of rows of one or more data tables within the first data set;
generate, for a second data set, a second graphical data set object comprising a graphical size that is based on a number of rows of one or more data tables within the second data set;
output, at a graphical user interface (GUI), the first graphical data set object and the second graphical data set object;
determine, based on a first user interaction, a score for one or more relationships between the first data set and the second data set, wherein the first user interaction comprises movement, via the GUI, of the first graphical data set object closer in position within the GUI to the second graphical data set object;
generate, based on the first user interaction, and based on the score for the one or more relationships satisfying a threshold, a graphical connector object that connects the first graphical data set object to the second graphical data set object within the GUI to indicate the score for the one or more relationships satisfies the threshold; and
output, via the GUI, the first graphical data set object and the second graphical data set object with the graphical connector object.
16. The non-transitory computer-readable medium of claim 15 , wherein the first data set and the second data set comprise one or more common attributes.
17. The non-transitory computer-readable medium of claim 15 , wherein the first data set and the second data set comprise one or more common values.
18. The non-transitory computer-readable medium of claim 15 , wherein the processor-executable instructions that cause the one or more processors to determine the score for the one or more relationships between the first data set and the second data set further cause the one or more processors to determine the score for the one or more relationships between the first data set and the second data set based on a profiling/classification parameter that comprises one or more of:
one or more ratios of unique values in one or more fields of data within the one or more data tables associated with each of the first data set and the second data set; one or more ratios of a size of an overlapping set of values from the one or more fields
one or more ratios of the size of the overlapping set of values from the one or more fields;
a history of user decisions made against the same or similar data;
metadata supplied by an external system, a developer, or an administrator;
a hint supplied by the developer or the administrator; or
a classification score that identifies a potential lookup table.
19. The non-transitory computer-readable medium of claim 15 , wherein the graphical size of each of the first graphical data set object and the second graphical data set object is based on logarithmic scaling applied to the number of rows of the one or more data tables within each data set.
20. The non-transitory computer-readable medium of claim 15 , wherein the processor-executable instructions further cause the one or more processors to output a data model based on the first data set and the second data set.
21. The non-transitory computer-readable medium of claim 15 , wherein the processor-executable instructions further cause the one or more processors to:
receive, via the GUI, a selection of the first graphical data set object; and
output, based on the selection, one or more of:
a preview of the first data set; or
a preview of a relationship of the one or more relationships.