Data analysis system and method
This disclosure relates to a system and method for data analysis. According to a first aspect, there is described a method, the method being performed using one or more processors, comprising: receiving one or more user inputs indicative of one or more relationships between data in a plurality of datasets; determining, based on the one or more user inputs, at least one object view for visualising the data in the plurality of datasets; generating, based on the one or more user inputs, metadata comprising: an object graph indicative of the one or more relationships between two or more of the plurality of datasets; and information identifying the at least one object view; and in response to a query relating to the plurality of datasets, using the metadata to determine how response data responding to the query should be provided.
1. A method, the method being performed using one or more processors, comprising:
accessing a plurality of datasets, wherein the plurality of datasets includes tables such that each table comprises rows representing objects and columns representing object properties;
receiving one or more user inputs indicative of one or more relationships between data in the plurality of datasets;
determining, based at least in part on the one or more user inputs, at least one object view for visualizing the data in the plurality of datasets;
determining a global property associated with a portion of data in at least two of the plurality of datasets, wherein the global property indicates one or more columns representing object properties that are associated with the portion of data;
generating, based at least in part on the one or more user inputs, metadata comprising:
an object graph indicative of the one or more relationships between two or more of the plurality of datasets;
the global property; and
information identifying the at least one object view; and
in response to a query relating to the plurality of datasets, using the metadata to determine how response data responding to the query should be provided, wherein the query is guided by the global property such that the query is applied to the portion of data associated with the global property.
2. The method of claim 1 , wherein the metadata further comprises a join between at least two datasets.
3. The method of claim 2 , wherein the global property is assigned to a portion of the data in at least two of the plurality of datasets.
4. The method of claim 3 , wherein determining how the response data should be provided comprises searching the plurality of datasets, wherein the search is guided by the global property.
5. The method of claim 4 , wherein the user input defines a global property that applies to the plurality of datasets.
6. The method of claim 5 , wherein the global property is determined using a method comprising:
receiving one or more statistics relating to the dataset;
comparing the one or more statistics to one or more threshold criteria; and
in response to one or more of the threshold criteria being met, providing a suggested global property to a user associated with the user input.
7. The method of claim 6 , wherein the user input comprises confirming the suggested global property provided to the user.
8. The method of claim 7 , wherein the one or more thresholds comprises at least one of: a proportion of datasets comprising an object type or property; a uniqueness score; cardinalities of object types or properties; or a title key.
9. The method of claim 8 , wherein the method further comprises inferring one or more joins between datasets based on the global property.
10. The method of claim 9 , wherein the global property is defined by one or more data formats or one or more data types.
11. The method of claim 10 , further comprising synchronizing the metadata with a metadata layer stored on a server.
12. The method of claim 11 , wherein the at least one object view comprises an object prominence.
13. A system comprising:
memory configured to store instructions; and
one or more processors in communication with the memory, the one or more processors configured to execute the instructions to cause the system to:
access a plurality of datasets, wherein the plurality of datasets includes tables such that each table comprises rows representing objects and columns representing object properties;
receive, via a user interface, one or more user inputs indicative of one or more relationships between data in the plurality of datasets;
determine, based at least in part on the one or more user inputs, at least one object view for visualizing the data in the plurality of datasets;
determining a global property associated with a portion of data in at least two of the plurality of datasets, wherein the global property indicates one or more columns representing object properties that are associated with the portion of data;
generate, based at least in part on the one or more user inputs, metadata comprising:
an object graph indicative of the one or more relationships between two or more of the plurality of datasets;
the global property; and
information identifying the at least one object view; and
in response to a query relating to the plurality of datasets, use the metadata to determine how response data responding to the query should be provided wherein the query is guided by the global property such that the query is applied to the portion of data associated with the global property.
14. The system of claim 13 , wherein the metadata further comprises a join between at least two datasets.
15. The system of claim 14 , wherein the global property is assigned to a portion of the data in at least two of the plurality of datasets.
16. The system of claim 15 , wherein determining how the response data should be provided comprises searching the plurality of datasets, wherein the search is guided by the global property.
17. The system of claim 16 , wherein the user input defines a global property that applies to the plurality of datasets.
18. The system of claim 17 , wherein the one or more processors are configured to execute the instructions to further cause the system to determine the global property by at least:
receiving one or more statistics relating to the dataset;
comparing the one or more statistics to one or more threshold criteria; and
in response to one or more of the threshold criteria being met, providing a suggested global property to a user associated with the user input.
19. The system of claim 18 , wherein the user input comprises confirming the suggested global property provided to the user.
20. A non-transitory computer readable medium having computer readable code stored thereon, the computer readable code, when executed by at least one processor, causing performance of the steps of:
accessing a plurality of datasets, wherein the plurality of datasets includes tables such that each table comprises rows representing objects and columns representing object properties;
receiving, via a user interface, one or more user inputs indicative of one or more relationships between data in the plurality of datasets;
determining, based at least in part on the one or more user inputs, at least one object view for visualizing the data in the plurality of datasets;
determining a global property associated with a portion of data in at least two of the plurality of datasets, wherein the global property indicates one or more columns representing object properties that are associated with the portion of data;
generating, based at least in part on the one or more user inputs, metadata comprising:
an object graph indicative of the one or more relationships between two or more of the plurality of datasets;
the global property; and
information identifying the at least one object view; and
in response to a query relating to the plurality of datasets, using the metadata to determine how response data responding to the query should be provided, wherein the query is guided by the global property such that the query is applied to the portion of data associated with the global property.