Generating differences for tuple attributes
A sequence of tuples, each having one or more attributes, is received at one of one or more processing elements operating on one or more processors. Each processing element may have one or more stream operators. A first stream operator may be identified as one that only processes an instance of a first attribute in a currently received tuple when a difference between an instance of the first attribute in a previously received tuple and the instance of the first attribute in the currently received tuple is outside of a difference threshold. A second stream operator may generate a difference attribute from a first instance of the first attribute in a first one of the received tuples and a second instance of the first attribute in a second one of the received tuples. The difference attribute may be transmitted from the second stream operator to the first stream operator.
1. A method for processing tuples on one or more computer processors, each tuple having one or more attributes, comprising:
receiving at one of one or more processing elements a sequence of two or more tuples to be processed by the one or more processing elements, each processing element having one or more stream operators;
identifying that a first stream operator only processes an instance of a first attribute in a currently received tuple when a difference between an instance of the first attribute in a previously received tuple and the instance of the first attribute in the currently received tuple is outside of a difference threshold;
generating, at a second stream operator, a difference attribute from a first instance of the first attribute and a second instance of the first attribute; and
transmitting the difference attribute from the second stream operator to the first stream operator.
2. The method of claim 1 , further comprising transmitting a reference instance of the first attribute to the first stream operator, wherein the reference instance of the first attribute is associated with an initial time.
3. The method of claim 2 , wherein the reference instance of the first attribute is substantially the same as the first instance of the first attribute.
4. The method of claim 2 , further comprising storing the reference instance of the first attribute and generating the second instance of the first attribute at the first stream operator, the second instance of the first attribute being generated from the difference attribute and the reference instance of the first attribute.
5. The method of claim 2 , further comprising storing the reference instance of the first attribute at the first stream operator, determining whether the difference attribute is outside of the difference threshold, and generating the second instance of the first attribute from the difference attribute and the reference instance of the first attribute when the difference attribute is outside of the difference threshold.
6. The method of claim 1 , further comprising determining that an inter-process communication path is operating outside of a latency threshold.
7. The method of claim 1 , further comprising determining that one or more instances of the first attribute are outside of a size threshold.
8. The method of claim 1 , further comprising determining that one or more instances of the difference attribute are inside of a variability threshold.
9. A method for processing tuples on one or more computer processors, each tuple having one or more attributes, comprising:
receiving at one of one or more processing elements a sequence of two or more tuples to be processed by the one or more processing elements, each processing element having one or more stream operators;
identifying that a first stream operator only processes an instance of a first attribute in a currently received tuple when a difference between an instance of the first attribute in a previously received tuple and the instance of the first attribute in the currently received tuple is outside of a difference threshold;
generating, at a second stream operator, a difference attribute from a first instance of the first attribute and a second instance of the first attribute;
determining whether the difference attribute is outside of the difference threshold; and
transmitting the difference attribute from the second stream operator to the first stream operator when the difference attribute is outside of the difference threshold.
10. The method of claim 9 , further comprising transmitting a reference instance of the first attribute to the first stream operator.
11. The method of claim 10 , wherein the reference instance of the first attribute is transmitted to the first stream operator in response to a request from the first stream operator.
12. The method of claim 10 , wherein the reference instance of the first attribute is substantially the same as the first instance of the first attribute.
13. The method of claim 10 , further comprising storing the reference instance of the first attribute at the first stream operator and generating the second instance of the first attribute from the difference attribute and the reference instance of the first attribute.
14. The method of claim 9 , wherein the one or more processing elements define an initial operator graph at a first time and the first stream operator is dynamically added to the initial operator graph at a second time subsequent to the first time.
15. The method of claim 9 , further comprising determining that an inter-process communication path is operating outside of a latency threshold.
16. The method of claim 9 , further comprising determining that one or more instances of the first attribute are outside of a size threshold.
17. The method of claim 9 , further comprising determining that one or more instances of the difference attribute are inside of a variability threshold.
18. A method for processing tuples on one or more computer processors, each tuple having one or more attributes, comprising:
receiving at one of one or more processing elements a sequence of two or more tuples to be processed by the one or more processing elements, each processing element having one or more stream operators;
identifying that a first stream operator only processes an instance of a first attribute in a currently received tuple when a difference between an instance of the first attribute in a previously received tuple and the instance of the first attribute in the currently received tuple is outside of a difference threshold;
determining, at a second stream operator, an estimate of resource usage required to determine a difference attribute from a first instance of the first attribute in a first one of the tuples received at the second stream operator and a second instance of the first attribute in a second one of the tuples received at the second stream operator;
determining whether the estimate of resource usage is outside of a resource usage threshold; and
transmitting one of the first instance of the first attribute of the first tuple or the second instance of the first attribute of the second tuple from the second stream operator to the first stream operator when the estimate of resource usage is outside the resource usage threshold.
19. The method of claim 18 , wherein when the estimate of resource usage is within the resource usage threshold, the method further comprises:
generating, at the second stream operator, the difference attribute;
determining whether the difference attribute is outside of the difference threshold; and
transmitting the difference attribute from the second stream operator to the first stream operator when the difference attribute is outside of the difference threshold.
20. The method of claim 18 , further comprising transmitting a third instance of the first attribute to the first stream operator in response to a request from the first stream operator.