Adaptive payload management
Embodiments of the invention relate to payload storage format for storing data in support of an aggregation function. As an input is subject to aggregation, the input is evaluated to ascertain a payload format for the aggregation. It is understood that there is more than one payload format. An evaluation of the aggregation key is a factor in the initial payload format. If the key is an addition to an existing aggregation, the evaluation considers changing the format of the payload to address processing and/or memory efficiency for the aggregation. The evaluation and the format change takes place dynamically so that the aggregation may continue.
1. A method comprising:
scanning an input data set containing keys and values;
aggregating the scanned input set on a select key and value;
evaluating the aggregation with respect to a payload storage format and selecting an initial format of an aggregation result based on an aggregation function;
storing an updated aggregation payload in the selected payload format; and
responsive to a subsequent aggregation of keys and values, dynamically re-evaluating the aggregation and selectively migrating the selected payload format to a new format based on the evaluation.
2. The method of claim 1 , wherein the aggregation evaluation includes examination of one or more payload aggregation formats.
3. The method of claim 2 , further comprising assessing a current payload format and evaluating efficiency of the assessed format for the aggregation, the assessment based on a factor selected from the group consisting of: memory space required of the assessed format from the aggregation, processing cost of adding one or more additional values to the current payload format, and numerical stability of the updated aggregation payload in the current payload format, and further comprising evaluating the selected factor with respect to the current aggregation and current resource availability.
4. The method of claim 3 , further comprising migrating payload content into an alternate format.
5. The method of claim 3 , further comprising adjusting a size of the payload format for accommodating the aggregation.
6. The method of claim 1 , wherein the selection of the initial format is based on a prior aggregation evaluation for one or more other aggregation keys on a query selected from the group consisting of: a present query and a prior query.
7. The method of claim 1 , further comprising creating a combined aggregate by merging two or more payload formats.
8. A computer program product for adaptive migration of an aggregate payload format, the computer program product comprising a computer readable storage device having program code embodied therewith, the program code executable by a processing unit to:
scan an input data set containing keys and values;
aggregate the scanned input set on a select key and value;
evaluate the aggregation with respect to a payload storage format and select an initial format of an aggregation result based on an aggregation function;
store an updated aggregation payload in the selected payload format; and
responsive to a subsequent aggregation of keys and values, dynamically re-evaluate the aggregation and selectively migrate the selected payload format to a new format based on the evaluation.
9. The computer program product of claim 8 , wherein the aggregation evaluation includes examination of one or more payload aggregation formats.
10. The computer program product of claim 9 , further comprising program code to assess a current payload format and evaluate efficiency of the assessed format for the aggregation, the assessment based on a factor selected from the group consisting of: memory space required of the assessed format from the aggregation, processing cost of adding one or more additional values to the current payload format, and numerical stability of the updated aggregation payload in the current payload format.
11. The computer program product of claim 10 , further comprising program code to migrate payload content into an alternate format.
12. The computer program product of claim 11 , further comprising program code to adjust a size of the payload format to accommodate the aggregation.
13. The computer program product of claim 8 , wherein the selection of the initial format is based on a prior aggregation evaluation for one or more other aggregation keys on a query selected from the group consisting of: a present query and a prior query.
14. The computer program product of claim 8 , further comprising program code to create a combined aggregate by merging two or more payload formats.
15. A computer system comprising:
a processing unit operatively coupled to memory;
an input data set stored at a first memory location, the input data set containing keys and values;
a tool in communication with the processing unit to adaptively migrate an aggregate payload format, including:
a director to scan the input data set and to aggregate the scanned data set on a select key and value;
the director to evaluate the aggregation with respect to a payload storage format and to select an initial format of an aggregation result based on an aggregation function;
the director to store an updated aggregation payload in the selected format at s second memory location; and
a manager in communication with the director, the manager to function in response to a subsequent aggregation of keys and values, including the manager to dynamically re-evaluate the aggregation and to selectively migrate the selected payload format to a new format based on the evaluation, wherein the migration includes a conversion of the selected payload format to the new format.
16. The computer system of claim 15 , wherein the aggregation evaluation performed by the director includes examination of one or more payload aggregation formats.
17. The computer system of claim 16 , further comprising the manager to assess a current payload format and evaluate efficiency of the assessed format for the aggregation, the assessment based on a factor selected from the group consisting of: memory space required of the assessed format from the aggregation, processing cost of adding one or more additional values to the current payload format, and numerical stability of the updated aggregation payload in the current payload format.
18. The computer system of claim 17 , further comprising the manager to migrate payload content into an alternate format.
19. The computer system of claim 17 , further comprising the manager to adjust a size of the payload format to accommodate the aggregation.
20. The computer system of claim 15 , further comprising the manager to create a combined aggregate by merging two or more payload formats.