Method and system for optimized delta encoding
View Patent ↗Methods and systems are provided for batch processing. A plurality of correlation parameters representing degrees of correlation between two or more data sets in a first batch of data sets may be determined, and an optimized correlation dependency graph may be generated according to the plurality of correlation parameters. A second batch of data sets may then be encoded according to the optimized correlation dependency graph. For example, the optimized correlation dependency graph may be used for performing a predetermined correlation encoding operation. The plurality of correlation parameters may be determined, for example, in accordance with one or more predetermined correlation metrics associated with a predetermined correlation encoding operation.
1. A method for batch processing, the method comprising:
determining a plurality of correlation parameters representing degrees of correlation between two or more data sets in a first batch of data sets;
generating an optimized correlation dependency graph according to said plurality of correlation parameters; and
encoding a second batch of data sets according to said optimized correlation dependency graph.
2. The method of claim 1 , comprising determining said plurality of correlation parameters in accordance with one or more predetermined correlation metrics associated with a predetermined correlation encoding operation.
3. The method of claim 1 , comprising determining an optimized data subdivision of said first batch of data sets.
4. The method of claim 1 , comprising determining an optimized correlation metric for determining said plurality of correlation parameters.
5. The method of claim 1 , wherein said optimized correlation dependency graph indicates a maximum of said plurality of correlation parameters.
6. The method of claim 1 , comprising performing correlation encoding on said second batch of data sets in a sequence determined by said optimized correlation dependency graph.
7. The method of claim 1 , comprising generating a plurality of correlation-encoded delta sets for a predetermined data set processing operation.
8. The method of claim 7 , wherein said predetermined data set processing operation comprises delta encoding.
9. The method of claim 7 , wherein said predetermined data set processing operation comprises correlation analysis of said plurality of correlation-encoded delta sets.
10. The method of claim 7 , wherein said predetermined data set processing operation comprises correlation optimization of said plurality of correlation-encoded delta sets.
11. A system for batch processing, the system comprising:
a processor that is operable to:
determine a plurality of correlation parameters representing degrees of correlation between two or more data sets in a first batch of data sets;
generate an optimized correlation dependency graph according to said plurality of correlation parameters; and
encode a second batch of data sets based on said optimized correlation dependency graph.
12. The system of claim 11 , wherein the processor is operable to determine said plurality of correlation parameters in accordance with one or more predetermined correlation metrics associated with a predetermined correlation encoding operation.
13. The system of claim 11 , wherein the processor is operable to determine an optimized data subdivision of said first batch of data sets.
14. The system of claim 11 , wherein the processor is operable to determine an optimized correlation metric for determining said plurality of correlation parameters.
15. The system of claim 11 , wherein the processor is operable to generate said optimized correlation dependency graph to indicate a maximum of said plurality of correlation parameters.
16. The system of claim 11 , wherein the processor is operable to perform correlation encoding on said second batch of data sets in a sequence determined by said optimized correlation dependency graph.
17. The system of claim 11 , wherein the processor is operable to generate a plurality of correlation-encoded delta sets for performing a predetermined data set processing operation.
18. The system of claim 17 , wherein said predetermined data set processing operation comprises delta encoding.
19. The system of claim 18 , wherein said predetermined data set processing operation comprises correlation analysis of said plurality of correlation-encoded delta sets.
20. The system of claim 18 , wherein said predetermined data set processing operation comprises correlation optimization of said plurality of correlation-encoded delta sets.