Signature-based cache optimization for data preparation

1. A system, comprising:

a data processing system comprising memory and one or more processors to:

receive, from a user interface, a first set of operations to be applied in a sequential order to one or more sets of data;

transform, by the first set of operations, a first set of data among the one or more sets of data, to generate a second set of data;

cache a representation of the second set of data to re-generate the second set of data;

cache a first signature corresponding at least in part to a first graph structure of the first set of operations, and the second set of data;

receive a specification of receive, from the user interface, a second set of operations to be applied in a sequential order to one or more sets of data;

generate a second signature corresponding at least in part to a second graph structure of the second set of operations;

regenerate the second set of data based on the cached representation, in response to determining a match between at least a portion of the first graph structure corresponding to the first signature and at least a portion of the second graph structure corresponding to the second signature; and

instruct the user interface to present output based at least in part on the regenerated second set of data.

2. The system of claim 1 wherein the first set of operations comprises at least one of clustering, joining, appending, sorting, uppercase, lowercase, filtering, deduplicating, grouping by, adding or removing columns, adding or removing rows, pivoting, or depivoting.

3. The system of claim 1 wherein the first signature comprises a hash corresponding to at least a portion of the first set of operations.

4. The system of claim 1 wherein the first signature comprises an order independent grouping of representations of at least a portion of the first set of operations.

5. The system of claim 1 wherein determining the match between the first signature and the second signature comprises determining an overlap between at least a portion of the first graph structure corresponding to the first operation signature and at least a portion of the second graph structure corresponding to the second operation signature.

6. The system of claim 1 wherein determining the match between the first signature and the second signature comprises:

manipulating the second signature to determine an equivalent operation signature; and

comparing the equivalent signature against the first signature.

7. The system of claim 6 wherein the equivalent signature is determined at least in part by performing an operator push down on the second signature.

8. The system of claim 1 wherein the representation of the second set of data comprises a data traversal program, and wherein the one or more processors are configured to re-generate the second set of data at least in part by executing the data traversal program.

9. The system of claim 1 wherein instructing the user interface to present output based at least in part on the re-generated second set of data comprises displaying, by the user interface, at least a portion of the re-generated second set of data.

10. The system of claim 1 wherein instructing the user interface to present output based at least in part on the re-generated second set of data comprises exporting at least a portion of the re-generated second set of data.

11. A method, comprising:

receiving, from a user interface, a first set of operations to be applied in a sequential order to one or more sets of data;

transforming, by the first set of operations, a first set of data among the one or more sets of data, to generate a resulting data set that is different from the one or more input sets of data;

caching a representation of the second set of data to re-generate the resulting data set;

caching a first signature corresponding at least in part to a first graph structure of the first set of operations, and the second set of data;

receiving, from the user interface, a second set of operations to be applied in a sequential order to one or more sets of data;

generating a second signature corresponding at least in part to a second graph structure of the second set of operations;

regenerating the second set of data based on the cached representation, in response to determining a match between at least a portion of the first graph structure corresponding to the first signature and at least a portion of the second graph structure corresponding to the second signature; and

instructing the user interface to present output based at least in part on the regenerated second set of data.

12. The method of claim 11 wherein the first set of operations comprises at least one of clustering, joining, appending, sorting, uppercase, lowercase, filtering, deduplicating, grouping by, adding or removing columns, adding or removing rows, pivoting, or depivoting.

13. The method of claim 11 wherein the first signature comprises a hash corresponding to at least a portion of the first set of operations.

14. The method of claim 11 wherein the first signature comprises an order independent grouping of representations of at least a portion of the first set of operations.

15. The method of claim 11 wherein determining the match between the first signature and the second signature comprises determining an overlap between at least a portion of the first graph structure corresponding to the first operation signature and at least a portion of the second graph structure corresponding to the second operation signature.

16. The method of claim 11 wherein determining the match between the first signature and the second operation comprises:

manipulating the second signature to determine an equivalent operation signature; and

comparing the equivalent signature against the first signature.

17. The method of claim 16 wherein the equivalent signature is determined at least in part by performing an operator push down on the second signature.

18. The method of claim 11 wherein the representation of the second set of data comprises a data traversal program, and wherein the one or more processors are configured to re-generate the second set of data set at least in part by executing the data traversal program.

19. The method of claim 11 wherein instructing the user interface to present output based at least in part on the re-generated second set of data comprises displaying, by the user interface, at least a portion of the re-generated second set of data.

20. The method of claim 11 wherein instructing the user interface to present output based at least in part on the re-generated second set of data comprises exporting at least a portion of the re-generated second set of data.

21. A computer program product embodied in a non-transitory computer readable storage medium and comprising one or more instructions executable by a processor to:

receiving, from a user interface, a first set of operations to be applied in a sequential order to one or more sets of data;

transforming, by the first set of operations, a first set of data among the one or more sets of data, to generate a resulting data set that is different from the one or more input sets of data;

caching a representation of the second set of data to re-generate the resulting data set;

caching a first signature corresponding at least in part to a first graph structure of the first set of operations, and the second set of data;

receiving, from the user interface, a second set of operations to be applied in a sequential order to one or more sets of data;

generating a second signature corresponding at least in part to a second graph structure of the second set of operations;

regenerating the second set of data based on the cached representation, in response to determining a match between at least a portion of the first graph structure corresponding to the first signature and at least a portion of the second graph structure corresponding to the second signature; and

instructing the user interface to present output based at least in part on the regenerated second set of data.