Handling missing data tuples in a streaming environment

1. A computer-implemented method for processing a stream of tuples, the method comprising:

establishing an operator graph for a plurality of processing elements that each include a respective one or more operators configured to perform a respective one or more operations on tuples of streaming data, wherein the operator graph defines at least one execution path through the operators of the plurality of processing elements, and wherein a first processing element in the operator graph includes a barrier operator that joins output of a plurality of upstream operators included in the one or more of the plurality of processing elements;

determining at least one timeout condition has occurred at the barrier operator;

generating, in response to the determining at least one timeout condition has occurred at the barrier operator, substitute data tuple values for missing tuple output data from at least one of the plurality of upstream operators;

generating a first barrier tuple output by joining the substitute data tuple values with received data tuple values from the plurality of upstream operators;

storing a copy of the received data tuple values;

receiving, at the barrier operator and after the storing of the copy of the received data tuple values, a set of one or more tuples;

determining the set of one or more tuples comprises the missing tuple output data; and

generating a second barrier tuple output by joining the missing tuple output data values with the copy of the received data tuple values.

2. The method of claim 1 , further comprising:

identifying, based on the determining the set of one or more tuples comprises the missing tuple output data, the first barrier tuple output at a downstream operator; and

invalidating the barrier tuple output.

3. The method of claim 2 , wherein the generating the first barrier tuple output further comprises assigning a unique identifier to the first barrier tuple output, and wherein the identifying the barrier tuple output at the downstream operator comprises identifying the unique identifier in the first barrier tuple output.

4. The method of claim 1 , wherein the generating the first barrier tuple output further comprises assigning a unique identifier to the first barrier tuple output, the method further comprising:

receiving, at a sink operator, the first barrier tuple output;

storing, at the sink operator, data from the first barrier tuple output;

receiving, at the sink operator, the second barrier tuple output; and

replacing the stored data from the first barrier tuple output with data from the second barrier tuple output.

5. The method of claim 4 , wherein the generating the first barrier tuple output further comprises assigning a unique identifier to the first barrier tuple output, and wherein the data from the first barrier tuple output is stored with the unique identifier, and wherein the generating the second barrier tuple output further comprises assigning the unique identifier to the second barrier tuple output; and wherein the replacing the stored data from the first barrier tuple output occurs in response to determining the unique identifier from the second barrier tuple output matches the unique identifier with the data from the first barrier tuple output.

6. A system for processing a stream of tuples comprising:

a plurality of processing elements to receive the stream of tuples;

two or more processors; and

a memory containing an application that, when executed, causes at least one of the two or more processors to perform a method comprising:

establishing an operator graph for the plurality of processing elements that each include a respective one or more operators configured to perform a respective one or more operations on tuples of streaming data, wherein the operator graph defines at least one execution path through the operators of the plurality of processing elements, and wherein a first processing element in the operator graph includes a barrier operator that joins output of a plurality of upstream operators included in the one or more of the plurality of processing elements;

determining at least one timeout condition has occurred at the barrier operator;

generating, in response to the determining at least one timeout condition has occurred at the barrier operator, substitute data tuple values for missing tuple output data from at least one of the plurality of upstream operators;

generating a first barrier tuple output by joining the substitute data tuple values with received data tuple values from the plurality of upstream operators;

storing a copy of the received data tuple values;

receiving, at the barrier operator and after the storing of the copy of the received data tuple values, a set of one or more tuples;

determining the set of one or more tuples comprises the missing tuple output data; and

generating a second barrier tuple output by joining the missing tuple output data values with the copy of the received data tuple values.

7. The system of claim 6 , wherein the method further comprises:

identifying, based on the determining the set of one or more tuples comprises the missing tuple output data, the first barrier tuple output at a downstream operator; and

invalidating the barrier tuple output.

8. The system of claim 7 , wherein the generating the first barrier tuple output further comprises assigning a unique identifier to the first barrier tuple output, and wherein the identifying the barrier tuple output at the downstream operator comprises identifying the unique identifier in the first barrier tuple output.

9. The system of claim 6 , wherein the generating the first barrier tuple output further comprises assigning a unique identifier to the first barrier tuple output, the method further comprising:

receiving, at a sink operator, the first barrier tuple output;

storing, at the sink operator, data from the first barrier tuple output;

receiving, at the sink operator, the second barrier tuple output; and

replacing the stored data from the first barrier tuple output with data from the second barrier tuple output.

10. The system of claim 9 , wherein the generating the first barrier tuple output further comprises assigning a unique identifier to the first barrier tuple output, and wherein the data from the first barrier tuple output is stored with the unique identifier, and wherein the generating the second barrier tuple output further comprises assigning the unique identifier to the second barrier tuple output; and wherein the replacing the stored data from the first barrier tuple output occurs in response to determining the unique identifier from the second barrier tuple output matches the unique identifier with the data from the first barrier tuple output.

11. A computer program product for processing a stream of tuples, the computer program product comprising a computer readable storage medium having program code embodied therewith, the program code executable by one or more processors to cause the one or more processors to perform a method comprising:

establishing an operator graph for a plurality of processing elements that each include a respective one or more operators configured to perform a respective one or more operations on tuples of streaming data, wherein the operator graph defines at least one execution path through the operators of the plurality of processing elements, and wherein a first processing element in the operator graph includes a barrier operator that joins output of a plurality of upstream operators included in the one or more of the plurality of processing elements;

determining at least one timeout condition has occurred at the barrier operator;

generating, in response to the determining at least one timeout condition has occurred at the barrier operator, substitute data tuple values for missing tuple output data from at least one of the plurality of upstream operators;

generating a first barrier tuple output by joining the substitute data tuple values with received data tuple values from the plurality of upstream operators;

storing a copy of the received data tuple values;

receiving, at the barrier operator and after the storing of the copy of the received data tuple values, a set of one or more tuples;

determining the set of one or more tuples comprises the missing tuple output data; and

generating a second barrier tuple output by joining the missing tuple output data values with the copy of the received data tuple values.

12. The computer program product of claim 11 , wherein the method further comprises:

identifying, based on the determining the set of one or more tuples comprises the missing tuple output data, the first barrier tuple output at a downstream operator; and

invalidating the barrier tuple output.

13. The computer program product of claim 12 , wherein the generating the first barrier tuple output further comprises assigning a unique identifier to the first barrier tuple output, and wherein the identifying the barrier tuple output at the downstream operator comprises identifying the unique identifier in the first barrier tuple output.

14. The computer program product of claim 11 , wherein generating the first barrier tuple output further comprises assigning a unique identifier to the first barrier tuple output, and wherein the method further comprises:

receiving, at a sink operator, the first barrier tuple output;

storing, at the sink operator, data from the first barrier tuple output;

receiving, at the sink operator, the second barrier tuple output; and

replacing the stored data from the first barrier tuple output with data from the second barrier tuple output.

15. The computer program product of claim 14 , wherein the generating the first barrier tuple output further comprises assigning a unique identifier to the first barrier tuple output, and wherein the data from the first barrier tuple output is stored with the unique identifier, and wherein the generating the second barrier tuple output further comprises assigning the unique identifier to the second barrier tuple output; and wherein the replacing the stored data from the first barrier tuple output occurs in response to determining the unique identifier from the second barrier tuple output matches the unique identifier with the data from the first barrier tuple output.