System and method for varying memory size in a data stream processing
The present disclosure relates to methods, systems, and computer-readable media for varying a memory size in a data stream processing while improving a connection degree sketch. Embodiments of the present disclosure may encode an input data by using an error coding technique to produce an encoded data, wherein the encoded data results in a modified memory size; generate a host connectivity using a set of parameters and applying a reverse sketching technique over the encoded data in order to obtain estimated encoded data; and decode the encoded data after the host connectivity is established using a decoding technique and obtaining an output data. The memory size of the output data may be proportional to the memory size of the input data.
1. A computer-implemented method for varying a memory size in a data stream processing while improving a connection degree sketch for identifying network traffic anomalies, the method comprising:
encoding an input data stream by using an error coding technique to produce an encoded data, wherein the encoded data results in a modified memory size;
generating a host connection degree sketch using a set of parameters and applying a reverse sketching technique over the encoded data in order to obtain estimated encoded data, wherein obtaining the estimated encoded data includes estimating in-degree and out-degree associated with one or more host in a network; and
decoding the encoded data after the host connection degree sketch is generated using a decoding technique and obtaining an output data, wherein a memory size of the output data is dependent on the memory size of the input data stream.
2. The method of claim 1 , wherein the input data comprises at least one of: an IP address, a plurality of TV programs, and a set of pairings of location identifiers and channel identifiers.
3. The method of claim 1 , wherein the error coding technique comprises at least one of: Hamming, Bose, Chaudhuri and Hocquenghem (BCH), Reed-Solomon, Low Density Parity-Check (WPC), Reed-Muller, and Convolutional codes.
4. The method of claim 1 , wherein the modified memory size of the encoded data is always greater than the memory size of the input data.
5. The method of claim 1 , wherein the set of parameters further comprises of number-of-rows and number-of-hash functions.
6. The method of claim 1 , wherein decoding permits detection and correction of one or more anomalies in the data stream.
7. The method of claim 1 , wherein improving the connection degree sketch by varying a memory size in a data stream processing comprises reducing the false positives and false negatives in the estimation of connection degree associated with the one or more hosts.
8. A system for varying a memory size in a data stream processing while improving a connection degree sketch for identifying network traffic anomalies, the system comprising:
a processor; and
a memory storing instructions that, when executed by the processor, cause the processor to perform a method, the method comprising:
encoding an input stream by using an error coding technique to produce an encoded data, wherein the encoded data results in a modified memory size;
generating a host connection, degree sketch using a set of parameters and applying a reverse sketching technique over the encoded data in order to obtain estimated encoded data wherein obtaining the estimated encoded data includes estimating the in-degree and out-degree associated with one or more hosts in a network; and
decoding the encoded data after the host connection degree sketch is generated using a decoding technique and obtaining an output data, wherein a memory size of the output data is dependent on the memory size of the input data stream.
9. The system of claim 8 , wherein the input data comprises at least one of: an IP address, a plurality of TV programs, and a set of pairings of location identifiers and channel identifiers.
10. The system of claim 8 , wherein the error coding technique comprises at least one of: Hamming, Bose, Chaudhuri and Hocquenghem (BCH), Reed-Solomon, Low Density Parity-Check (LDPC), Reed-Muller, and Convolutional codes.
11. The system of claim 8 , wherein the modified memory size of the encoded data is always greater than the memory size of the input data.
12. The system of claim 8 , wherein decoding permits detection and correction of one or more anomalies in the data stream.
13. The method of claim 8 , wherein improving the connection degree sketch by varying a memory size in a data stream processing comprises reducing the false positives and false negatives in the estimation of connection degree associated with the one or more hosts.
14. A non-transitory, computer-readable medium, having stored thereon instructions that, when executed by a processor, cause the processor to perform a method, the method comprising:
encoding an input data stream by using an error coding technique to produce an encoded data, wherein the encoded data results in a modified memory size;
generating a host connection degree sketch using a set of parameters and applying a reverse sketching technique over the encoded data in order to obtain estimated encoded data wherein obtaining the estimated encoded data includes estimating the in-degree and out-degree associated with one or more hosts in a network; and
decoding the encoded data after the host connection degree sketch is generated using a decoding technique and obtaining an output data, wherein a memory size of the output data is dependent on the memory size of the input data stream.