Transformation in tabular data cleaning tool
A system including first computer memory storing a full data set representable in rows and columns, a second computer memory storing executable instructions, and processors configured to execute the instructions to cause presentation of data of the full data set on a display including columns of data each having data fields, receive user input identifying a column of the data set, determine items to modify in information in the data fields of the identified column, generate and cause display of an indication of a proposed change action to modify the determined items, and in response to a user input indicating a selection of the indication of the proposed change action, update the presentation of the data based on the change action to modify information displayed in the data fields of the identified column of the data, and store a log of the change action.
1. A system, comprising:
one or more computer processors configured to execute computer-executable instructions to cause the system to at least:
generate and cause display on a display device a plurality of columns representing a subset of data that is a portion of a large first data set, the columns having rows of data, the displayed columns showing only a portion of the rows of data of the first data set, the first data set including data field information;
receive user input selecting a portion of the displayed columns;
generate and cause display of data metrics, of the first data set, for data field information in the selected portion of the columns for all the rows including rows of the selected columns that are not displayed, the data metrics being predetermined based on the type of data fields in the identified portion of the subset of data;
generate and cause display of a change action to modify data field information of the first data set; and
in response to a selection of a displayed change action, cause update of the data metrics on the display device, the update of the data metrics reflecting changes that would be made in the first data set caused by the selected change action without modifying the first data set.
2. The system of claim 1 , wherein the one or more computer hardware processors are further configured to execute the computer-executable instructions to iteratively:
receive user input identifying a portion of the subset of data;
determine data field information to modify for at least some of the data fields of the identified portion of the subset of data; and
generate and cause display of an indication of a change action to modify the determined data field information in the first data set.
3. The system of claim 2 , wherein the one or more computer hardware processors are further configured to execute the computer-executable instructions to iteratively:
receive user input indicating a selection of a change action, and
store a log of the change action.
4. The system of claim 3 , wherein the one or more computer hardware processors are further configured to execute the computer-executable instructions to:
access the log;
apply each change action in the log to the first data set; and
save a second data set, the second data set including modifications made to the first data set.
5. The system of claim 3 , wherein the one or more computer hardware processors are further configured to execute the computer-executable instructions to:
access the log;
apply each change action stored in the log to a second data set; and
save a third data set, the third data set including modifications made to the second data set by the change actions.
6. The system of claim 1 , wherein the change action includes modifying the data field information to at least one of: changing the spelling of a word; changing the case of letters; deleting a space; adding a space; deleting a period, comma, semi-colon, or colon; or adding a period, comma, semi-colon, or colon.
7. The system of claim 1 , wherein the change action comprises searching a plurality of the data fields to identify data fields that include first information, and in the identified data fields replace the first information with second information.
8. The system of claim 1 , wherein the data fields are configured to be of a data type, and wherein the change action includes changing the data type of at least one data field.
9. The system of claim 1 , wherein the change action comprises concatenating at least one alphanumeric character or punctuation to information in a plurality of data fields.
10. The system of claim 1 , wherein generate and cause display of the data metrics comprises determining errors in information in the data fields of the identified portion of the subset of data by analyzing the information in the data fields according to associated criteria to determine erroneous information in the data fields.
11. The system of claim 1 , wherein the first data set is representable in a tabular rows and columns format, the columns include hundreds of rows of data, and the identified portion of the subset of data is a column of data.
12. The system of claim 1 , wherein the one or more computer hardware processors are further configured to execute the computer-executable instructions to determine data field information to modify including filtering information in data fields to identify selected data fields to modify the information therein.
13. The system of claim 1 , wherein the one or more computer hardware processors are further configured to execute the computer-executable instructions to determine data field information to modify including filtering information in data fields to determine selected data fields, and causing presentation of the information in the selected data fields on a user interface.
14. A method of preparing tabular representable data for further processing, the method comprising:
generate and cause display on a display device a plurality of columns representing a subset of data that is a portion of a large first data set, the columns having rows of data, the displayed columns showing only a portion of the rows of data of the first data set, the first data set including data field information;
receiving user input selecting a portion of the displayed columns;
generating and causing display of data metrics, of the first data set, for data field information in the selected portion of the columns for all the rows including rows of the selected columns that are not displayed, the data metrics being predetermined based on the type of data fields in the identified portion of the subset of data;
generating and causing display of a change action to modify data field information of the first data set; and
in response to a selection of a displayed change action, cause update of the data metrics on the display device, the update of the data metrics reflecting changes that would be made in the first data set caused by the selected change action without modifying the first data set.
15. The method of claim 14 , further comprising:
iteratively,
receiving a user input identifying a portion of the subset of data;
determining data field information to modify for at least some of the data fields of the identified portion of the subset of data; and
generating and causing display of an indication of a change action to modify the determined data field information.
16. The method of claim 15 , further comprising:
in response to a user input indicating a selection of the change action, storing a log of the change action.
17. The method of claim 16 , further comprising:
accessing the log;
applying each change action stored in the log to the first data set; and
saving a second data set, the second data set including the modifications made to the first data set.
18. The method of claim 16 , further comprising:
accessing the log;
applying each change action stored in the log to a second data set; and
saving a third data set, the third second data set including modifications made to the second data set by applying each change action.
19. The method of claim 14 , wherein the first data set is representable in a tabular rows and columns format, wherein the columns include hundreds of rows of data, wherein the identified portion of the subset of data is a column of data, and wherein the change action includes modifying the data field information in the column of data by at least one of: changing the spelling of a word; changing the case of a letter; deleting a space; adding a space; deleting a period, comma, semi-colon, or colon; or adding a period, comma, semi-colon, or colon.
20. The method of claim 14 , wherein generating and causing display of the data metrics comprises determining errors in information in the data fields of the identified portion of the subset of data by analyzing the information in the data fields according to associated criteria to determine erroneous information in the data fields.