Differential processing of data streams based on protocols
This disclosure relates to the processing of data streams. More specifically, application of particular protocols to a stream and a detection analysis facilitate a selective, reliable and efficient transmission of pertinent stream data to destination addresses.
1. A method comprising:
receiving a first data stream of a plurality of data streams, the first data stream:
comprising a first plurality of data elements, and
having been transmitted by a first device of a plurality of devices;
assigning the first data stream of the plurality of data streams to a first processing node of a plurality of processing nodes comprising a set of one or more stream processors;
receiving a second data stream of the plurality of data streams, the second data stream:
comprising a second plurality of data elements, and
having been transmitted by the first device or a second device of the plurality of devices;
assigning the second data stream of the plurality of data streams to a second processing node of the plurality of processing nodes;
detecting that a first report protocol applies to the first data stream and/or the second data stream;
determining whether data elements in the first data stream and/or the second data stream include a particular composite from a set of one or more composites specified in the first report protocol having values consistent with one or more values specified in the first report protocol;
generating a first report at least partially by:
retrieving a first set of information for each data element in the first data stream and/or the second data stream for which it was determined to include a value associated with the particular composite that is consistent with the one or more values specified in the first report protocol, and
transmitting the first set of information to at least one destination address associated with a subset of one or more devices of the plurality of devices;
storing one or more identifiers of the first report in a log;
generating a second report at least partially by:
retrieving a second set of information for each data element in one or more of the first data stream, the second data stream, and/or a third data stream for which it was determined to include a value associated with a different composite that is consistent with one or more values specified in a second report protocol; and
associating one or more identifiers of the second report with a data element corresponding to the first report; and
storing the second report with the one or more identifiers of the second report associated with the data element corresponding to the first report.
2. The method as recited in claim 1 , where the one or more identifiers of the second report further identify the data element corresponding to the first report.
3. The method as recited in claim 2 , where the one or more identifiers of the second report further identify a time of transmission corresponding to the transmitting.
4. The method as recited in claim 3 , where the one or more identifiers of the second report further identify the at least one destination address or a destination corresponding to the at least one destination address.
5. The method as recited in claim 2 , further comprising:
generating a third report at least partially by:
retrieving a third set of information for each data element in one or more of the first data stream, the second data stream, and/or a third data stream for which it was determined to include a value associated with the particular composite that is consistent with the one or more values specified in the first report protocol, and
transmitting the third set of information; and
storing one or more identifiers of the third report in the log, where the one or more identifiers of the third report identify the data element corresponding to the first report.
6. The method as recited in claim 2 , further comprising:
generating the second report at least partially by:
transmitting the second set of information; and
storing the one or more identifiers of the second report in the log.
7. The method as recited in claim 6 , where the first report protocol specifies a first set of one or more reporting triggers, and the transmitting the first set of information is responsive to at least one trigger from the first set of one or more reporting triggers.
8. A system comprising:
one or more processing devices communicatively coupled to memory, the one or more processing devices to perform operations comprising:
processing a first data stream of a plurality of data streams, the first data stream:
comprising a first plurality of data elements, and
having been transmitted by a first device of a plurality of devices;
assigning the first data stream of the plurality of data streams to a first processing node of a plurality of processing nodes comprising a set of one or more stream processors;
processing a second data stream of the plurality of data streams, the second data stream:
comprising a second plurality of data elements, and
having been transmitted by the first device or a second device of the plurality of devices;
assigning the second data stream of the plurality of data streams to a second processing node of the plurality of processing nodes;
detecting that a first report protocol applies to the first data stream and/or the second data stream;
determining whether data elements in the first data stream and/or the second data stream include a particular composite from a set of one or more composites specified in the first report protocol having values consistent with one or more values specified in the first report protocol;
generating a first report at least partially by:
retrieving a first set of information for each data element in the first data stream and/or the second data stream for which it was determined to include a value associated with the particular composite that is consistent with the one or more values specified in the first report protocol, and
transmitting the first set of information to at least one destination address associated with a subset of one or more devices of the plurality of devices;
causing storing one or more identifiers of the first report in a log;
generating a second report at least partially by:
retrieving a second set of information for each data element in one or more of the first data stream, the second data stream, and/or a third data stream for which it was determined to include a value associated with a different composite that is consistent with one or more values specified in a second report protocol; and
associating one or more identifiers of the second report with a data element corresponding to the first report; and
causing storing the second report with the one or more identifiers of the second report associated with the data element corresponding to the first report.
9. The system as recited in claim 8 , where the one or more identifiers of the second report further identify the data element corresponding to the first report.
10. The system as recited in claim 9 , where the one or more identifiers of the second report further identify a time of transmission corresponding to the transmitting.
11. The system as recited in claim 10 , where the one or more identifiers of the second report further identify the at least one destination address or a destination corresponding to the at least one destination address.
12. The system as recited in claim 9 , the operations further comprising:
generating a third report at least partially by:
retrieving a third set of information for each data element in one or more of the first data stream, the second data stream, and/or a third data stream for which it was determined to include a value associated with the particular composite that is consistent with the one or more values specified in the first report protocol, and
transmitting the third set of information; and
causing storing one or more identifiers of the third report in the log, where the one or more identifiers of the third report identify the data element corresponding to the first report.
13. The system as recited in claim 9 , the operations further comprising:
generating the second report at least partially by:
transmitting the second set of information; and
causing storing the one or more identifiers of the second report in the log.
14. The system as recited in claim 13 , where the first report protocol specifies a first set of one or more reporting triggers, and the transmitting the first set of information is responsive to at least one trigger from the first set of one or more reporting triggers.
15. One or more non-transitory, machine-readable media having machine-readable instructions thereon which, when executed by one or more processing devices, cause the one or more processing devices to perform operations comprising:
processing a first data stream of a plurality of data streams, the first data stream:
comprising a first plurality of data elements, and
having been transmitted by a first device of a plurality of devices;
assigning the first data stream of the plurality of data streams to a first processing node of a plurality of processing nodes comprising a set of one or more stream processors;
processing a second data stream of the plurality of data streams, the second data stream:
comprising a second plurality of data elements, and
having been transmitted by the first device or a second device of the plurality of devices;
assigning the second data stream of the plurality of data streams to a second processing node of the plurality of processing nodes;
detecting that a first report protocol applies to the first data stream and/or the second data stream;
determining whether data elements in the first data stream and/or the second data stream include a particular composite from a set of one or more composites specified in the first report protocol having values consistent with one or more values specified in the first report protocol;
generating a first report at least partially by:
retrieving a first set of information for each data element in the first data stream and/or the second data stream for which it was determined to include a value associated with the particular composite that is consistent with the one or more values specified in the first report protocol, and
transmitting the first set of information to at least one destination address associated with a subset of one or more devices of the plurality of devices;
causing storing one or more identifiers of the first report in a log;
generating a second report at least partially by:
retrieving a second set of information for each data element in one or more of the first data stream, the second data stream, and/or a third data stream for which it was determined to include a value associated with a different composite that is consistent with one or more values specified in a second report protocol; and
associating one or more identifiers of the second report with a data element corresponding to the first report; and
causing storing the second report with the one or more identifiers of the second report associated with the data element corresponding to the first report.
16. The one or more non-transitory, machine-readable media as recited in claim 15 , where the one or more identifiers of the second report further identify the data element corresponding to the first report.
17. The one or more non-transitory, machine-readable media as recited in claim 16 , where the one or more identifiers of the second report further identify a time of transmission corresponding to the transmitting.
18. The one or more non-transitory, machine-readable media as recited in claim 17 , where the one or more identifiers of the second report further identify the at least one destination address or a destination corresponding to the at least one destination address.
19. The one or more non-transitory, machine-readable media as recited in claim 16 , the operations further comprising:
generating a third report at least partially by:
retrieving a third set of information for each data element in one or more of the first data stream, the second data stream, and/or a third data stream for which it was determined to include a value associated with the particular composite that is consistent with the one or more values specified in the first report protocol, and
transmitting the third set of information; and
causing storing one or more identifiers of the third report in the log, where the one or more identifiers of the third report identify the data element corresponding to the first report.
20. The one or more non-transitory, machine-readable media as recited in claim 16 , the operations further comprising:
generating the second report at least partially by:
transmitting the second set of information; and
causing storing the one or more identifiers of the second report in the log.