MONITORING NETWORK TRAFFIC TO DETERMINE SIMILAR CONTENT
In an embodiment, a method monitors a plurality of data streams passing through a router in the connectivity service provider environment, and for each of the data streams, periodically samples packets at the router. The method further generates a stream signature based at least on the payload of the sampled packets. The method further includes, for each generated stream signature, attaching information to the stream signature. Such information may, for example, include time-stamp information for the stream signature, or an identification of the router. The method may further comprise storing the stream signatures corresponding to the data streams in a database. The stored stream signatures may be compared to determine matching stream signatures. Matching signatures may identify data streams that carry identical or similar content.
1 . A system comprising:
at least one processor; and
memory storing instructions that, when executed by the at least one processor, causes the system to perform a set of operations, the set of operations comprising:
sampling packets of a first data stream received at a first router of a plurality of routers in the connectivity service provider environment;
generating a stream signature of the first data stream based at least on a subset of the sampled packets, the stream signature comprising an identification of the first router at which the sampled packets were received;
determining, based at least in part on the identification of the router in the stream signature, that the first data stream is similar to a second data stream relayed through a second router different than the first router; and
rerouting, based on the determining, at least the first data stream within the connectivity service provider environment.
2 . The system of claim 1 , wherein the subset of sampled packets is generated by filtering the sampled packets to exclude at least one packet based on a determination that the at least one packet does not change a stream signature for the first data stream.
3 . The system of claim 2 , wherein the stream signature is generated without parsing a payload of the filtered packets.
4 . The system of claim 2 , wherein the stream signature is generated further based at least on a source address and a destination address of at least one packet of the filtered packets.
5 . The system of claim 1 , wherein the stream signature is generated without parsing a payload of the subset of sampled packets.
6 . The system of claim 1 , wherein the stream signature is generated further based on statistical information about the first data stream comprising at least one of:
mean of data rate;
variance of data rate;
skewness of data rate;
minimum data rate; or
maximum data rate.
7 . The system of claim 1 , wherein the first data stream is sampled according to a predetermined time interval, and wherein the first data stream is sampled for a predetermined time period.
8 . A method for identifying similar data streams in a connectivity service provider environment, the method comprising:
sampling packets of a first data stream received at a first router of a plurality of routers in the connectivity service provider environment;
generating a stream signature of the first data stream based at least on a subset of the sampled packets, the stream signature comprising an identification of the first router at which the sampled packets were received;
determining, based at least in part on the identification of the router in the stream signature, that the first data stream is similar to a second data stream relayed through a second router different than the first router; and
rerouting, based on the determining, at least the first data stream within the connectivity service provider environment.
9 . The method of claim 8 , wherein the subset of sampled packets is generated by filtering the sampled packets to exclude at least one packet based on a determination that the at least one packet does not change a stream signature for the first data stream.
10 . The method of claim 9 , wherein the stream signature is generated without parsing a payload of the filtered packets.
11 . The method of claim 9 , wherein the stream signature is generated further based at least on a source address and a destination address of at least one packet of the filtered packets.
12 . The method of claim 8 , wherein the stream signature is generated without parsing a payload of the subset of sampled packets.
13 . The method of claim 8 , wherein the stream signature is generated further based on statistical information about the first data stream comprising at least one of:
mean of data rate;
variance of data rate;
skewness of data rate;
minimum data rate; or
maximum data rate.
14 . The method of claim 8 , wherein the first data stream is sampled according to a predetermined time interval, and wherein the first data stream is sampled for a predetermined time period.