System and method for providing scalable flow monitoring in a data center fabric
Disclosed is a method that includes calculating, at a collector receiving a data flow and via a hashing algorithm, all possible hashes associated with at least one virtual attribute associated with the data flow to yield resultant hash values. Based on the resultant hash values, the method includes computing a multicast address group and multicasting the data flow to n leafs based on the multicast address group. At respective other collectors, the method includes filtering received sub-flows of the data flow based on the resultant hashes, wherein if a respective hash is owned by a collector, the respective collector accepts and saves the sub-flow in a local switch collector database. A scalable, distributed netflow is possible with the ability to respond to queries for fabric-level netflow statistics even on virtual constructs.
1. A method for creating a virtual netflow collector to distribute netflow packet collections across switches in a network, the method comprising:
distributing netflow packet collectors across all switches in a network fabric of switches to yield a virtual netflow collector, the netflow packet collectors configured to collect data from one or more capturing agents;
calculating a first hash at a source switch of an incoming packet on each possible combination of a 5-tuple IP packet header;
calculating a second hash at the source switch of the incoming packet on each possible combination of at least one virtual attribute of the network fabric of switches;
creating an N-Tuple flow from the first hash and the second hash;
exporting the N-Tuple flow to the virtual netflow collector;
mapping, via a virtual extensible local area network multicast address group, the virtual netflow collector to one or more physical netflow collectors; and
providing a visualization response, in response to a query regarding the N-Tuple flow at the virtual netflow collector.
2. The method of claim 1 , wherein each of the switches is configured to function as a netflow collector.
3. The method of claim 1 , further comprising:
receiving a query at the virtual netflow collector regarding the N-Tuple flow; and
presenting a response to the query based on the mapping of the virtual netflow collector to the one or more physical netflow collectors.
4. The method of claim 3 , wherein the response comprises a visualization response.
5. The method of claim 3 , wherein the query uses at least one of the first hash and the second hash.
6. The method of claim 1 , wherein aggregated statistics for a given virtual attribute end up in a same physical collector instance according to the mapping of the virtual netflow collector to the one or more physical netflow collectors.
7. The method of claim 1 , wherein the at least one virtual attribute comprises one or more of a tenant, a virtual routing and forwarding object, an endpoint group, a bridge domain, a subnet, a contract, an application, or a filter.
8. A system for creating a virtual netflow collector to distribute netflow packet collections across switches in a network, the system comprising:
at least one processor; and
a computer-readable storage device storing instructions which, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
distributing netflow packet collectors across all switches in a network fabric of switches to yield a virtual netflow collector, the netflow packet collectors configured to collect data from one or more capturing agents;
calculating a first hash at a source switch of an incoming packet on each possible combination of a 5-tuple IP packet header;
calculating a second hash at the source switch of the incoming packet on each possible combination of at least one virtual attribute of the network fabric of switches;
creating an N-Tuple flow from the first hash and the second hash;
exporting the N-Tuple flow to the virtual netflow collector;
mapping, via a virtual extensible local area network multicast address group, the virtual netflow collector to one or more physical netflow collectors; and
providing a visualization response, in response to a query regarding the N-Tuple flow at the virtual netflow collector.
9. The system of claim 8 , wherein each of the switches is configured to function as a netflow collector.
10. The system of claim 8 , wherein the computer-readable storage device stores additional instructions which, when executed by the at least one processor, cause the at least one processor to perform operations further comprising:
receiving a query at the virtual netflow collector regarding the N-Tuple flow; and
presenting a response to the query based on the mapping of the virtual netflow collector to the one or more physical netflow collectors.
11. The system of claim 10 , wherein the response comprises a visualization response.
12. The system of claim 10 , wherein the query uses at least one of the first hash and the second hash.
13. The system of claim 8 , wherein aggregated statistics for a given virtual attribute end up in a same physical collector instance according to the mapping of the virtual netflow collector to the one or more physical netflow collectors.
14. The system of claim 8 , wherein the at least one virtual attribute comprises one or more of a tenant, a virtual routing and forwarding object, an endpoint group, a bridge domain, a subnet, a contract, an application, or a filter.
15. A non-transitory computer readable medium, for creating a virtual netflow collector to distribute netflow packet collections across switches in a network, storing instructions which, when executed by at least one processor, causes the at least one processor to perform operations comprising:
distributing netflow packet collectors across all switches in a network fabric of switches to yield a virtual netflow collector, the netflow packet collectors configured to collect data from one or more capturing agents;
calculating a first hash at a source switch of an incoming packet on each possible combination of a 5-tuple IP packet header;
calculating a second hash at the source switch of the incoming packet on each possible combination of at least one virtual attribute of the network fabric of switches;
creating an N-Tuple flow from the first hash and the second hash;
exporting the N-Tuple flow to the virtual netflow collector;
mapping, via a virtual extensible local area network multicast address group, the virtual netflow collector to one or more physical netflow collectors; and
providing a visualization response, in response to a query regarding the N-Tuple flow at the virtual netflow collector.
16. The non-transitory computer readable medium of claim 15 , wherein each of the switches is configured to function as a netflow collector.
17. The non-transitory computer readable medium of claim 15 , storing additional instructions which, when executed by the at least one processor, cause the at least one processor to perform operations further comprising:
receiving a query at the virtual netflow collector regarding the N-Tuple flow; and
presenting a response to the query based on the mapping of the virtual netflow collector to the one or more physical netflow collectors.
18. The non-transitory computer readable medium 17 , wherein the response comprises a visualization response.
19. The non-transitory computer readable medium 17 , wherein the query uses at least one of the first hash and the second hash.
20. The non-transitory computer readable medium of claim 15 , wherein aggregated statistics for a given virtual attribute end up in a same physical collector instance according to the mapping of the virtual netflow collector to the one or more physical netflow collectors.