Smart column selection for table operations in data preparation
A computer implemented method of providing suggestions of related columns for a table operation in a data management system is provided. In one embodiment, the method includes storing, by a processor, a table having a plurality of rows and a plurality of columns, metadata associated with the plurality of columns, and data stored within the plurality of rows and the plurality of columns; receiving, by a processor, a table operation selected by a system or user; processing, by a processor, at least one of the data stored in the plurality of columns and the metadata associated with plurality columns based on a type of the selected table operation to determine at least one relationship between at least two columns of the plurality of columns; and generating, by the processor, dialog data that presents the at least one relationship between the at least two columns to a system or user for use in the table operation.
1. A computer implemented method of providing suggestions of related columns for a table operation in a data management system, the method comprising:
storing, by a processor, a table having a plurality of rows and a plurality of columns, metadata associated with the plurality of columns, and data stored within the plurality of rows and the plurality of columns, wherein the metadata includes at least one of a data type of each column, or a size of each column;
receiving a table operation selected by a system or user;
processing, by a processor, the metadata associated with the plurality of columns based on a type of the selected table operation to determine at least one relationship between at least two columns of the plurality of columns of the table, wherein the processing comprises selecting a relationship determination method from a plurality of relationship determination methods based on the type of the selected table operation, computing a similarity score between the at least two columns based on the metadata and the selected relationship determination method, and determining the at least one relationship based on a comparison of the similarity score to a threshold; and
generating, by the processor, dialog data that presents the at least one relationship between the at least two columns to a system or user for use in the table operation.
2. The method of claim 1 , further comprising determining other table operations that can be performed on the at least one relationship and wherein the dialog data presents the other table operations.
3. The method of claim 1 , wherein the processing is based on at least one of an average, a median, a min-max, a data histogram, frequent values, and data completeness of the data stored in the plurality of columns, and wherein the processing is performed on the data stored in the plurality of columns.
4. The method of claim 1 , wherein the table operation is a pivot operation.
5. The method of claim 1 , wherein the table operation is at least one of a pivot operation, a join operation, a merge operation, and an append operation.
6. The method of claim 1 , wherein the relationship determination method is selected based on the type of the selected table operation.
7. The method of claim 1 , wherein the relationship determination method is a rule based determination method.
8. The method of claim 1 , wherein the relationship determination method is a rule based method that makes use of at least one of predefined rule sets and automatic rule inference.
9. A computer implemented system of providing suggestions of related columns for a table operation in a data management system, the system comprising:
a non-transitory computer readable medium configured to store, by a processor, a table having a plurality of rows and a plurality of columns, metadata associated with the plurality of columns, and data stored within the plurality of rows and the plurality of columns, wherein the metadata includes at least one of a data type of each column, or a size of each column;
a processor configured to receive a table operation selected by a system or user, process the metadata associated with plurality columns based on a type of the selected table operation to determine at least one relationship between at least two columns of the plurality of columns of the table, wherein the processing comprises selecting a relationship determination method from a plurality of relationship determination methods based on the type of the selected table operation, computing a similarity score between the at least two columns based on the metadata and the selected relationship determination method, and determining the at least one relationship based on a comparison of the similarity score to a threshold, and generate dialog data that presents the at least one relationship between the at least two columns to a system or user for use in the table operation.
10. The system of claim 9 , wherein the processor is further configured to determine other table operations that can be performed on the at least one relationship and wherein the dialog data presents the other table operations.
11. The system of claim 9 , wherein the processor is configured to process based on at least one of an average, a median, a min-max, a data histogram, frequent values, and data completeness of the data stored in the plurality of columns, and wherein the processor is configured to process the data stored in the plurality of columns.
12. The system of claim 9 , wherein the table operation is at least one of a pivot operation, a join operation, a merge operation, and an append operation.
13. The system of claim 12 , wherein the relationship determination method is selected based on the type of the selected table operation.
14. The system of claim 9 , wherein the relationship determination method is a rule based determination method.
15. The system of claim 9 , wherein the relationship determination method is a rule based method that makes use of at least one of predefined rule sets and automatic rule inference.
16. An enterprise social networking system comprising one or more processors and non-transient computer readable media coupled to the one or more processors, the non-transient computer readable media embodying programming instructions configurable to perform a method, the method comprising:
storing, by a processor, a table having a plurality of rows and a plurality of columns, metadata associated with the plurality of columns, and data stored within the plurality of rows and the plurality of columns, wherein the metadata includes at least one of a data type of each column, or a size of each column;
receiving, by a processor, a table operation selected by a system or a user;
processing, by a processor, the metadata associated with plurality columns based on a type of the selected table operation to determine at least one relationship between at least two columns of the plurality of columns of the table, wherein the processing comprises selecting a relationship determination method from a plurality of relationship determination methods based on the type of the selected table operation, computing a similarity score between the at least two columns based on the metadata and the selected relationship determination method, and determining the at least one relationship based on a comparison of the similarity score to a threshold; and
generating, by the processor, dialog data that presents the at least one relationship between the at least two columns to a system or user for use in the table operation.