Elastic modification of application instances in a network visibility infrastructure
Introduced here are network visibility platforms having total processing capacity that can be dynamically varied in response to determining how much network traffic is currently under consideration. A visibility platform can include one or more network appliances, each of which includes at least one instance of an application configured to process data packets. Rather than forward all traffic to a single application instance for processing, the traffic can instead be distributed amongst a pool of application instances to collectively ensure that no data packets are dropped due to over-congestion. Moreover, the visibility platform can be designed such that application instances are elastically added/removed, as necessary, based on the volume of traffic currently under consideration.
1. A method comprising:
comparing an amount of traffic output by a plurality of virtual machines to available processing capacity of a number of existing instances of an application that is configured to process the traffic; and
modifying, based on a result of said comparing, the number of existing instances of the application so as to accommodate variations in the amount of traffic in real time.
2. The method of claim 1 , wherein said modifying comprises:
creating a new instance of the application responsive to a determination that the amount of traffic exceeds a predetermined amount of the available processing capacity.
3. The method of claim 1 , wherein said modifying comprises:
removing an existing instance of the application responsive to a determination that the amount of traffic falls below a predetermined amount of the available processing capacity.
4. The method of claim 1 , wherein each instance of the application has a fixed processing capacity that limits how much traffic can be handled at one time.
5. The method of claim 1 , wherein each instance of the application executes on a different network appliance.
6. The method of claim 1 , further comprising:
causing the traffic to be distributed amongst the existing instances of the application to prevent dropping of data packets.
7. The method of claim 1 , further comprising:
acquiring, via an application programming interface, the traffic that is indicative of data packets output by the plurality of virtual machines hosted on a multi-tenant cloud computing platform;
wherein said comparing is performed repeatedly as the traffic is acquired.
8. A non-transitory computer-readable medium with instructions stored thereon that, when executed by a processor of a computing device, cause the computing device to perform operations comprising:
determining that traffic indicative of data packets handled by a plurality of virtual machines hosted on a multi-tenant cloud computing platform exceeds a capacity limitation of a first instance of an application;
creating, in real time responsive to said determining, a second instance of the application; and
separating the traffic into (i) a first group of data packets to be processed by the first instance of the application and (ii) a second group of data packets to be processed by the second instance of the application.
9. The non-transitory computer-readable medium of claim 8 , wherein said separating comprises:
generating a hash value for each data packet included in the traffic to produce a plurality of hash values,
accessing a data structure that maps the plurality of hash values amongst a plurality of instances of the application,
determining that each data packet in the first group of data packets corresponds to an entry in the data structure that specifies the first instance of the application, and
determining that each data packet in the second group of data packets corresponds to an entry in the data structure that specifies the second instance of the application.
10. The non-transitory computer-readable medium of claim 9 , wherein each instance of the application is executed in a different node of a visibility fabric.
11. The non-transitory computer-readable medium of claim 8 , wherein the operations further comprise:
forwarding (i) the first group of data packets to the first instance of the application and (ii) the second group of data packets to the second instance of the application;
forming a filtered stream of data packets by combining at least a portion of the first group of data packets with at least a portion of the second group of data packets.
12. The non-transitory computer-readable medium of claim 11 ,
wherein the at least a portion of the first group of data packets are those data packets that were not filtered by the first instance of the application, and
wherein the at least a portion of the second group of data packets are those data packets that were not filtered by the second instance of the application.
13. The non-transitory computer-readable medium of claim 11 , wherein the operations further comprise:
forwarding the filtered stream of data packets to an egress port for transmission toward a destination.
14. The non-transitory computer-readable medium of claim 13 , wherein the destination is the plurality of virtual machines.
15. The non-transitory computer-readable medium of claim 8 , wherein the plurality of virtual machines are associated with a single tenant of the multi-tenant cloud computing platform.
16. The non-transitory computer-readable medium of claim 8 , wherein the first group of data packets includes all data packets in the traffic that match a first criterion specified in a first flow map, and wherein the second group of data packets includes all data packets in the traffic that match a second criterion specified in a second flow map.