Differential processing of data streams based on protocols
This disclosure relates to the processing of data streams. More specifically, application of particular protocols to a stream and a detection analysis facilitate a selective, reliable and efficient transmission of pertinent stream data to destination addresses.
1. A system comprising:
one or more processing devices communicatively coupled to memory, the one or more processing devices to perform operations comprising:
processing one or more data streams, each data stream of the one or more data streams having been transmitted by a source and including a plurality of data elements;
for a data stream of the one or more data streams:
detecting that a report protocol applies to the data stream based on the source of the data stream; and
assigning the data stream to one or more processing nodes based on the report protocol;
for each data element of the plurality of data elements in the data stream:
determining, during streaming of the data stream, whether the data element includes a particular composite as specified in the report protocol;
when it is determined that the data element includes the particular composite, determining, during streaming of the data stream, whether the data element includes a value associated with the particular composite that is consistent with one or more values specified in the report protocol; and
when it is determined that the data element includes a value associated with the particular composite that is consistent with the one or more values, caching information associated with the data element;
detecting one or more reporting triggers that are specified in the report protocol;
retrieving, from the cache and for each data element of the plurality of data elements for which it was determined that the data element includes the value associated with the particular composite that is consistent with the one or more values, the information that is associated with the data element;
reducing a degree of detail in at least some of the information in accordance with a detail-conversion process specified by a rule, the detail-conversion process resulting in detail-converted information; and
transmitting at least the detail-converted information to a destination address.
2. The system as recited in claim 1 , where the detail-conversion process resulting in the detail-converted information comprises a reduction in information precision.
3. The system as recited in claim 2 , the operations further comprising:
generating each of a first communication and a second communication based at least in part on the information, each of the first communication and second communication comprising at least some of the information associated with the data element of the plurality of data elements that included the value associated with the particular composite that is consistent with the one or more values;
wherein the transmitting at least the detail-converted information to the destination address corresponds to transmitting the first communication to the destination address; and
transmitting the second communication to a second destination address.
4. The system as recited in claim 3 , wherein the second communication comprises the information and does not comprise the detail-converted information.
5. The system as recited in claim 3 , wherein the first communication and the second communication differ with respect to a degree of identification of a subject associated with at least one of the data elements.
6. The system as recited in claim 3 , wherein the detail-conversion process resulting in the detail-converted information comprises transforming the at least some of the information into the detail-converted information.
7. The system as recited in claim 3 , wherein the detail-conversion process resulting in the detail-converted information comprises categorizing the at least some of the information.
8. A method comprising:
receiving one or more data streams, each data stream of the one or more data streams having been transmitted by a source and including a plurality of data elements;
for a data stream of the one or more data streams:
detecting that a report protocol applies to the data stream based on the source of the data stream; and
assigning the data stream to one or more processing nodes based on the report protocol;
for each data element of the plurality of data elements in the data stream:
determining, during streaming of the data stream, whether the data element includes a particular composite as specified in the report protocol;
when it is determined that the data element includes the particular composite, determining, during streaming of the data stream, whether the data element includes a value associated with the particular composite that is consistent with one or more values specified in the report protocol; and
when it is determined that the data element includes a value associated with the particular composite that is consistent with the one or more values, caching information associated with the data element;
detecting one or more reporting triggers that are specified in the report protocol;
retrieving, from the cache and for each data element of the plurality of data elements for which it was determined that the data element includes the value associated with the particular composite that is consistent with the one or more values, the information that is associated with the data element;
reducing a degree of detail in at least some of the information in accordance with a detail-conversion process specified by a rule, the detail-conversion process resulting in detail-converted information; and
transmitting at least the detail-converted information to a destination address.
9. The method as recited in claim 3 , where the detail-conversion process resulting in the detail-converted information comprises a reduction in information precision.
10. The method as recited in claim 9 , further comprising:
generating each of a first communication and a second communication based at least in part on the information, each of the first communication and second communication comprising at least some of the information associated with the data element of the plurality of data elements that included the value associated with the particular composite that is consistent with the one or more values;
wherein the transmitting at least the detail-converted information to the destination address corresponds to transmitting the first communication to the destination address; and
transmitting the second communication to a second destination address.
11. The method as recited in claim 10 , wherein the second communication comprises the information and does not comprise the detail-converted information.
12. The method as recited in claim 10 , wherein the first communication and the second communication differ with respect to a degree of identification of a subject associated with at least one of the data elements.
13. The method as recited in claim 10 , wherein the detail-conversion process resulting in the detail-converted information comprises transforming the at least some of the information into the detail-converted information.
14. The method as recited in claim 10 , wherein the detail-conversion process resulting in the detail-converted information comprises categorizing the at least some of the information.
15. One or more non-transitory, machine-readable media having machine-readable instructions thereon which, when executed by one or more processing devices, cause the one or more processing devices to perform operations comprising:
processing one or more data streams, each data stream of the one or more data streams having been transmitted by a source and including a plurality of data elements;
for a data stream of the one or more data streams:
detecting that a report protocol applies to the data stream based on the source of the data stream; and
assigning the data stream to one or more processing nodes based on the report protocol;
for each data element of the plurality of data elements in the data stream:
determining, during streaming of the data stream, whether the data element includes a particular composite as specified in the report protocol;
when it is determined that the data element includes the particular composite, determining, during streaming of the data stream, whether the data element includes a value associated with the particular composite that is consistent with one or more values specified in the report protocol; and
when it is determined that the data element includes a value associated with the particular composite that is consistent with the one or more values, caching information associated with the data element;
detecting one or more reporting triggers that are specified in the report protocol;
retrieving, from the cache and for each data element of the plurality of data elements for which it was determined that the data element includes the value associated with the particular composite that is consistent with the one or more values, the information that is associated with the data element;
reducing a degree of detail in at least some of the information in accordance with a detail-conversion process specified by a rule, the detail-conversion process resulting in detail-converted information; and
transmitting at least the detail-converted information to a destination address.
16. The one or more non-transitory, machine-readable media as recited in claim 15 , where the detail-conversion process resulting in the detail-converted information comprises a reduction in information precision.
17. The one or more non-transitory, machine-readable media as recited in claim 16 , the operations further comprising:
generating each of a first communication and a second communication based at least in part on the information, each of the first communication and second communication comprising at least some of the information associated with the data element of the plurality of data elements that included the value associated with the particular composite that is consistent with the one or more values;
wherein the transmitting at least the detail-converted information to the destination address corresponds to transmitting the first communication to the destination address; and
transmitting the second communication to a second destination address.
18. The one or more non-transitory, machine-readable media as recited in claim 17 , wherein the second communication comprises the information and does not comprise the detail-converted information.
19. The one or more non-transitory, machine-readable media as recited in claim 17 , wherein the first communication and the second communication differ with respect to a degree of identification of a subject associated with at least one of the data elements.
20. The one or more non-transitory, machine-readable media as recited in claim 17 , wherein the detail-conversion process resulting in the detail-converted information comprises transforming the at least some of the information into the detail-converted information.