Methods and systems for data wrangling

1. A method for data wrangling, comprising:

determining one or more data operations to be performed on a multi-dimensional dataset;

determining at least one operation type associated with the one or more data operations to be performed;

determining a plurality of scripts and a plurality of sub-tables based on the at least one operation type;

extracting the sub-tables from the multi-dimensional dataset;

providing the plurality of scripts and the plurality of sub-tables to a data wrangling process, the data wrangling process processing each of the plurality of sub-tables in parallel using the scripts to determine a processed plurality of sub-tables;

receiving completion message(s) from the data wrangling process, the completion message(s) including the processed plurality of sub-tables; and

re-joining the processed plurality of sub-tables to the multi-dimensional dataset,

wherein the re-joining the processed plurality of sub-tables to the multi-dimensional dataset comprises:

extracting the processed plurality of sub-tables from the completion message(s); and

updating tables of the multi-dimensional dataset based on the processed plurality of sub-tables.

2. The method of claim 1 , wherein the data wrangling process comprises:

transmitting the scripts and plurality of sub-tables to a server, the server processing the plurality of sub-tables in parallel using the scripts and plurality of sub-tables to determine the processed plurality of sub-tables; and

receiving the completion message(s) from the server.

3. The method of claim 1 , wherein determining the plurality of scripts and the plurality of sub-tables based on the operation type further comprises:

splitting the multi-dimensional dataset into tables;

determining groupings of operations based on columns and/or rows of the tables targeted by the data operations;

extracting the plurality of sub-tables from the tables based on the groupings; and

determining scripts for each of the plurality of sub-tables based on the groupings.

4. The method of claim 1 , wherein determining the plurality of scripts and the plurality of sub-tables based on the operation type further comprises:

splitting the multi-dimensional dataset into tables;

determining groupings of operations based on columns and/or rows of the tables targeted by the data operations;

extracting the plurality of sub-tables from the tables based on the groupings; and

determining scripts for each of the plurality of sub-tables based on the groupings, and

wherein determining groupings of operations further comprises:

determining at least one operation of the one or more data operations to be performed on a column or row; and

determining at least two operations of the one or more data operations to be performed on adjacent columns or rows, and/or

determining a plurality of operations of the one or more data operations to be performed on columns or rows within a threshold number of columns or rows from each other.

5. The method of claim 1 , wherein determining the plurality of scripts and the plurality of sub-tables based on the operation type further comprises:

splitting the multi-dimensional dataset into tables;

determining groupings of operations based on columns and/or rows of the tables targeted by the data operations;

extracting the plurality of sub-tables from the tables based on the groupings; and

determining scripts for each of the plurality of sub-tables based on the groupings, and

wherein the extracting the plurality of sub-tables includes:

extracting columns or rows associated with a grouping based on the columns or rows targeted by the groupings of the operations.

6. The method of claim 1 , further comprising, before determining the one or more data operations:

receiving imported data; and

generating an initial multi-dimensional dataset based on the imported data and/or determining an existing multi-dimensional dataset to be updated based on the imported data, the initial multi-dimensional dataset or the existing multi-dimensional dataset being the multi-dimensional dataset upon which the one or more data operations are performed.

7. A system for data wrangling, the system comprising:

at least one memory storing instructions; and

at least one processor executing the instructions to perform operations, the operations including:

determining one or more data operations to be performed on a multi-dimensional dataset;

determining at least one operation type associated with the one or more data operations to be performed;

determining a plurality of scripts and a plurality of sub-tables based on the at least one operation type;

extracting the sub-tables from the multi-dimensional dataset;

providing the plurality of scripts and the plurality of sub-tables to a data wrangling process, the data wrangling process processing each of the plurality of sub-tables in parallel using the scripts to determine a processed plurality of sub-tables;

receiving completion message(s) from the data wrangling process, the completion message(s) including the processed plurality of sub-tables; and

re-joining the processed plurality of sub-tables to the multi-dimensional dataset,

wherein the re-joining the processed plurality of sub-tables to the multi-dimensional dataset comprises:

extracting the processed plurality of sub-tables from the completion message(s); and

updating tables of the multi-dimensional dataset based on the processed plurality of sub-tables.

8. The system of claim 7 , wherein the data wrangling process comprises:

transmitting the scripts and plurality of sub-tables to a server, the server processing the plurality of sub-tables in parallel using the scripts and plurality of sub-tables to determine the processed plurality of sub-tables; and

receiving the completion message(s) from the server.

9. The system of claim 7 , wherein determining the plurality of scripts and the plurality of sub-tables based on the operation type further comprises:

splitting the multi-dimensional dataset into tables;

determining groupings of operations based on columns and/or rows of the tables targeted by the data operations;

extracting the plurality of sub-tables from the tables based on the groupings; and

determining scripts for each of the plurality of sub-tables based on the groupings.

10. The system of claim 7 , wherein determining the plurality of scripts and the plurality of sub-tables based on the operation type further comprises:

splitting the multi-dimensional dataset into tables;

determining groupings of operations based on columns and/or rows of the tables targeted by the data operations;

extracting the plurality of sub-tables from the tables based on the groupings; and

determining scripts for each of the plurality of sub-tables based on the groupings, and

wherein determining groupings of operations further comprises:

determining at least one operation of the one or more data operations to be performed on a column or row; and

determining at least two operations of the one or more data operations to be performed on adjacent columns or rows, and/or

determining a plurality of operations of the one or more data operations to be performed on columns or rows within a threshold number of columns or rows from each other.

11. The system of claim 7 , wherein determining the plurality of scripts and the plurality of sub-tables based on the operation type further comprises:

splitting the multi-dimensional dataset into tables;

determining groupings of operations based on columns and/or rows of the tables targeted by the data operations;

extracting the plurality of sub-tables from the tables based on the groupings; and

determining scripts for each of the plurality of sub-tables based on the groupings, and

wherein the extracting the plurality of sub-tables includes:

extracting columns or rows associated with a grouping based on the columns or rows targeted by the groupings of the operations.

12. The system of claim 7 , the operations further comprising, before determining the one or more data operations:

receiving imported data; and

generating an initial multi-dimensional dataset based on the imported data and/or determining an existing multi-dimensional dataset to be updated based on the imported data, the initial multi-dimensional dataset or the existing multi-dimensional dataset being the multi-dimensional dataset upon which the one or more data operations are performed.

13. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations data wrangling, the operations comprising:

determining one or more data operations to be performed on a multi-dimensional dataset;

determining at least one operation type associated with the one or more data operations to be performed;

determining a plurality of scripts and a plurality of sub-tables based on the at least one operation type;

extracting the sub-tables from the multi-dimensional dataset;

providing the plurality of scripts and the plurality of sub-tables to a data wrangling process, the data wrangling process processing each of the plurality of sub-tables in parallel using the scripts to determine a processed plurality of sub-tables;

receiving completion message(s) from the data wrangling process, the completion message(s) including the processed plurality of sub-tables; and

re-joining the processed plurality of sub-tables to the multi-dimensional dataset,

wherein the re-joining the processed plurality of sub-tables to the multi-dimensional dataset comprises:

extracting the processed plurality of sub-tables from the completion message(s); and

updating tables of the multi-dimensional dataset based on the processed plurality of sub-tables.

14. The non-transitory computer-readable medium of claim 13 , wherein the data wrangling process comprises:

transmitting the scripts and plurality of sub-tables to a server, the server processing the plurality of sub-tables in parallel using the scripts and plurality of sub-tables to determine the processed plurality of sub-tables; and

receiving the completion message(s) from the server, the completion message(s) including the processed plurality of sub-tables; and

re-joining the processed plurality of sub-tables to the multi-dimensional dataset.

15. The non-transitory computer-readable medium of claim 13 , wherein determining the plurality of scripts and the plurality of sub-tables based on the operation type further comprises:

splitting the multi-dimensional dataset into tables;

determining groupings of operations based on columns and/or rows of the tables targeted by the data operations;

extracting the plurality of sub-tables from the tables based on the groupings; and

determining scripts for each of the plurality of sub-tables based on the groupings.

16. The non-transitory computer-readable medium of claim 13 , wherein determining the plurality of scripts and the plurality of sub-tables based on the operation type further comprises:

splitting the multi-dimensional dataset into tables;

determining groupings of operations based on columns and/or rows of the tables targeted by the data operations;

extracting the plurality of sub-tables from the tables based on the groupings; and

determining scripts for each of the plurality of sub-tables based on the groupings, and

wherein determining groupings of operations further comprises:

determining at least one operation of the one or more data operations to be performed on a column or row; and

determining at least two operations of the one or more data operations to be performed on adjacent columns or rows, and/or

determining a plurality of operations of the one or more data operations to be performed on columns or rows within a threshold number of columns or rows from each other.

17. The non-transitory computer-readable medium of claim 13 , wherein determining the plurality of scripts and the plurality of sub-tables based on the operation type further comprises:

splitting the multi-dimensional dataset into tables;

determining groupings of operations based on columns and/or rows of the tables targeted by the data operations;

extracting the plurality of sub-tables from the tables based on the groupings; and

determining scripts for each of the plurality of sub-tables based on the groupings, and

wherein the extracting the sub-tables includes:

extracting columns or rows associated with a grouping based on the columns or rows targeted by the groupings of the operations.