Methods and apparatus for detection of hierarchical heavy hitters
An efficient streaming method and apparatus for detecting hierarchical heavy hitters from massive data streams is disclosed. In one embodiment, the method enables near real time detection of anomaly behavior in networks.
1. A method for detecting at least one hierarchical heavy hitter from a stream of packets, comprising:
receiving at least one packet from said stream of packets;
associating at least one key with at least one field of said at least one packet;
applying an adaptive trie data structure, where each node of said adaptive trie data structure is associated with one of said at least one key; and
using said adaptive trie data structure to determine said at least one hierarchical heavy hitter, wherein said using said adaptive trie data structure to determine said at least one hierarchical heavy hitter comprises:
reconstructing a volume for each node this is an internal node;
estimating missed traffic for each of said internal node; and
determining said at least one hierarchical heavy hitter in accordance with a combination of said reconstructed volume and said estimated missed traffic.
2. The method of claim 1 , wherein said estimating missed traffic comprises using at least one of: a copy-all method, a no-copy method, and or a splitting method.
3. A computer-readable medium having stored thereon a plurality of instructions, the plurality of instructions including instructions which, when executed by a processor, cause the processor to perform the steps of a method for detecting at least one hierarchical heavy hitter from a stream of packets, comprising:
receiving at least one packet from said stream of packets;
associating at least one key with at least one field of said at least one packet;
applying an adaptive trie data structure, where each node of said adaptive trie data structure is associated with one of said at least one key; and
using said adaptive trie data structure to determine said at least one hierarchical heavy hitter, wherein said using said adaptive trie data structure to determine said at least one hierarchical heavy hitter comprises:
reconstructing a volume for each node this is an internal node;
estimating missed traffic for each of said internal node; and
determining said at least one hierarchical heavy hitter in accordance with a combination of said reconstructed volume and said estimated missed traffic.
4. The computer-readable medium of claim 3 , wherein said estimating missed traffic comprises using at least one of: a copy-all method, a no-copy method, or a splitting method.