Data analysis system and method
This disclosure relates to a system and method for data analysis. According to a first aspect, there is described a method, the method being performed using one or more processors, comprising: receiving one or more user inputs indicative of one or more relationships between data in a plurality of datasets; determining, based on the one or more user inputs, at least one object view for visualizing the data in the plurality of datasets; generating, based on the one or more user inputs, metadata comprising: an object graph indicative of the one or more relationships between two or more of the plurality of datasets; and information identifying the at least one object view; and in response to a query relating to the plurality of datasets, using the metadata to determine how response data responding to the query should be provided.
1. A method, the method being performed using one or more processors, comprising:
accessing a plurality of datasets, wherein the plurality of datasets includes tables such that each table comprises rows representing objects and columns representing properties;
determining a global property indicating one or more columns that are associated with a portion of data in two or more of the plurality of datasets, wherein determining the global property comprises at least determining that a threshold criterion is met based at least in part on one or more statistics that are associated with at least one of the plurality of datasets;
generating, based at least in part on one or more user inputs, metadata comprising:
the global property; and
information for visualizing the data in the two or more of the plurality of datasets; and
in response to a query relating to the plurality of datasets, using the metadata to determine how response data responding to the query should be provided.
2. The method of claim 1 , wherein the metadata further comprises a join between at least two of the plurality of datasets.
3. The method of claim 1 , wherein the global property is assigned to a portion of the data in at least two of the plurality of datasets.
4. The method of claim 1 , wherein determining how the response data should be provided comprises searching the plurality of datasets, wherein the searching is guided by the global property.
5. The method of claim 1 , wherein determining the global property comprises:
receiving the one or more statistics, wherein the determining that the threshold criterion is met further comprises comparing the one or more statistics to the threshold criterion; and
in response to the threshold criterion being met, providing a suggested global property to a user associated with the one or more user inputs.
6. The method of claim 5 , wherein determining the global property further comprises:
receiving a confirmation of the suggested global property.
7. The method of claim 5 , wherein the threshold criterion comprises at least one of: a proportion of the plurality of datasets comprising an object type or property; a uniqueness score; cardinalities of object types or properties; or a title key.
8. The method of claim 1 , further comprising:
inferring one or more joins between each of the plurality of datasets based on the global property.
9. The method of claim 1 , wherein the global property is defined by one or more data formats or one or more data types.
10. The method of claim 1 , further comprising:
synchronizing the metadata with a metadata layer stored on a server.
11. The method of claim 1 , wherein the at least one object view comprises an object prominence.
12. A system comprising:
memory configured to store instructions; and
one or more processors in communication with the memory, the one or more processors configured to execute the instructions to cause the system to:
access a plurality of datasets, wherein the plurality of datasets includes tables such that each table comprises rows representing objects and columns representing properties;
determine a global property indicating one or more columns that are associated with a portion of data in two or more of the plurality of datasets, wherein determining the global property comprises at least determining that a threshold criterion is met based at least in part on one or more statistics that are associated with at least one of the plurality of datasets;
generate, based at least in part on one or more user inputs, metadata comprising:
the global property; and
information for visualizing the data in the two or more of the plurality of datasets; and
in response to a query relating to the plurality of datasets, use the metadata to determine how response data responding to the query should be provided.
13. The system of claim 12 , wherein the metadata further comprises a join between at least two of the plurality of datasets.
14. The system of claim 12 , wherein the global property is assigned to a portion of the data in at least two of the plurality of datasets.
15. The system of claim 12 , wherein determining how the response data should be provided comprises searching the plurality of datasets, wherein the searching is guided by the global property.
16. The system of claim 12 , wherein determining the global property comprises:
receiving the one or more statistics, wherein the determining that the threshold criterion is met is performed by comparing the one or more statistics to the threshold criterion; and
in response to the threshold criterion being met, providing a suggested global property to a user associated with the one or more user inputs.
17. The system of claim 12 , wherein the system is further configured to:
infer one or more joins between each of the plurality of datasets based on the global property.
18. A non-transitory computer readable medium having computer readable code stored thereon, the computer readable code, when executed by at least one processor, causing performance of the steps of:
accessing a plurality of datasets, wherein the plurality of datasets includes tables such that each table comprises rows representing objects and columns representing properties;
determining a global property indicating one or more columns that are associated with a portion of data in two or more of the plurality of datasets, wherein determining the global property comprises at least determining that a threshold criterion is met based at least in part on one or more statistics that are associated with at least one of the plurality of datasets;
generating, based at least in part on one or more user inputs, metadata comprising:
the global property; and
information for visualizing the data in the two or more of the plurality of datasets; and
in response to a query relating to the plurality of datasets, using the metadata to determine how response data responding to the query should be provided.
19. The non-transitory computer readable medium of claim 18 , wherein the metadata further comprises a join between at least two of the plurality of datasets.
20. The non-transitory computer readable medium of claim 18 , wherein the global property is assigned to a portion of the data in at least two of the plurality of datasets.