Tracking tuples to reduce redundancy in a graph
View Patent ↗A stream of tuples can assigned identifiers to tuples to store only the nonduplicative tuples. In a streams processing environment, a stream application actor like an operator can receive a series of tuples, process them, and output another series of tuples. Each of the tuples can be assigned a tuple identifier. The tuple identifier can tag the tuple as associated with the operator. Another operator can receive the tuples, identify the duplicative tuples, and store only the nonduplicative tuples.
1. A computer implemented method for processing a stream of tuples, wherein the stream of tuples are to be processed by a plurality of processing elements operating on one or more computer processors, each processing element having one or more stream operators, wherein one or more of the stream operators include code configured to output tuples to one or more other stream operators, the method comprising:
receiving a first series of tuples at a first stream application actor, the first stream application actor configured to perform one or more operations on the first series of tuples and to output a second series of tuples;
assigning, to each tuple in the second series of tuples, a tuple identifier, the tuple identifier associating the first stream application actor to each tuple in the second series of tuples, wherein the tuple identifier comprises identification information relating each tuple in the second series of tuples to one or more sibling tuples;
receiving, by a second stream application actor, the second series of tuples, the second stream application actor comprising a windowing operator, the windowing operator having a window memory;
identifying, based on the assigned tuple identifiers and a set of parameters, a set of duplicative tuples in the second series of tuples; the set of duplicative tuples comprising one or more tuples that are duplicative in view of tuples outside the set of duplicative tuples and in the second series of tuples, and wherein the duplicative tuples are identified using the set of parameters, wherein the set of parameters define sibling tuples as duplicative; and
storing, in the window memory of the second stream application actor, a set of nonduplicative tuples for later processing, the window memory of the second stream application actor configured to store tuples, the set of nonduplicative tuples comprising-tuples remaining in the second series of tuples after a removal of the set of duplicative tuples therefrom.
2. The method of claim 1 , wherein the first stream application actor and the second stream application actor are processing elements.
3. The method of claim 1 , wherein the first stream application actor and the second stream application actor are stream operators.
4. The method of claim 1 , wherein the set of duplicate tuples are identical tuples.
5. The method of claim 1 , further comprising:
processing, in response to the identifying, before the storing, and by the second stream application actor, the second series of tuples.
6. The method of claim 1 , wherein the tuple identifier comprises identification information from a processing element.
7. The method of claim 1 , wherein the tuple identifier comprises identification information from the first stream application actor.
8. A computer system for processing a stream of tuples, wherein the stream of tuples are to be processed by a plurality of processing elements operating on one or more computer processors, each processing element having one or more stream operators, wherein one or more of the stream operators include code configured to output tuples to one or more other stream operators, the computer system comprising:
at least one processor circuit configured to:
receive a first series of tuples at a first stream application actor, the first stream application actor configured to perform one or more operations on the first series of tuples and to output a second series of tuples;
assign, to each tuple in the second series of tuples, a tuple identifier, the tuple identifier associating the first stream application actor to each tuple in the second series of tuples, wherein the tuple identifier comprises identification information relating each tuple in the second series of tuples to one or more sibling tuples;
receive, by a second stream application actor, the second series of tuples, the second stream application actor comprising a windowing operator, the windowing operator having a window memory;
identify, based on the assigned tuple identifiers and a set of parameters, a set of duplicative tuples in the second series of tuples; the set of duplicative tuples comprising one or more tuples that are duplicative in view of tuples outside the set of duplicative tuples and in the second series of tuples, and wherein the duplicative tuples are identified using the set of parameters, wherein the set of parameters define sibling tuples as duplicative; and
store, in the window memory of the second stream application actor, a set of nonduplicative tuples, for later processing, the window memory of the second stream application actor configured to store tuples, the set of nonduplicative tuples comprising tuples remaining in the second series of tuples after a removal of the set of duplicative tuples therefrom.
9. The computer system of claim 8 , wherein the first stream application actor and the second stream application actor are processing elements.
10. The computer system of claim 8 , wherein the first stream application actor and the second stream application actor are stream operators.
11. The computer system of claim 8 , wherein the set of duplicate tuples are identical tuples.
12. The computer system of claim 8 , wherein the at least one processor circuit is further configured to:
process, in response to the identifying, before the storing, and by the second stream application actor, the second series of tuples.
13. The computer system of claim 8 , wherein the tuple identifier comprises identification information from a processing element.
14. The computer system of claim 8 , wherein the tuple identifier comprises identification information from the first stream application actor.
15. A computer program product for processing a stream of tuples, wherein the stream of tuples are to be processed by a plurality of processing elements operating on one or more computer processors, each processing element having one or more stream operators, wherein one or more of the stream operators include code configured to output tuples to one or more other stream operators, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions executable by a computer processing circuit to cause the circuit to perform a method comprising:
receiving a first series of tuples at a first stream application actor, the first stream application actor configured to perform one or more operations on the first series of tuples and to output a second series of tuples;
assigning, to each tuple in the second series of tuples, a tuple identifier, the tuple identifier associating the first stream application actor to each tuple in the second series of tuples, wherein the tuple identifier comprises identification information relating each tuple in the second series of tuples to one or more sibling tuples;
receiving, by a second stream application actor, the second series of tuples, the second stream application actor comprising a windowing operator, the windowing operator having a window memory;
identifying, based on the assigned tuple identifiers and a set of parameters, a set of duplicative tuples in the second series of tuples; the set of duplicative tuples comprising one or more tuples that are duplicative in view of tuples outside the set of duplicative tuples and in the second series of tuples, and wherein the duplicative tuples are identified using the set of parameters, wherein the set of parameters define sibling tuples as duplicative; and
storing, in the window memory of the second stream application actor, a set of nonduplicative tuples, for later processing, the window memory of the second stream application actor configured to store tuples, the set of nonduplicative tuples comprising tuples remaining in the second series of tuples after a removal of the set of duplicative tuples therefrom.
16. The computer program product of claim 15 , wherein the method further comprises:
processing, in response to the identifying, before the storing, and by the second stream application actor, the second series of tuples.
17. The computer program product of claim 15 , wherein the tuple identifier comprises identification information from the first stream application actor.