Service level enforcement in distributed system using data package injection
Methods and systems for managing the operation of a deployment are provided. The deployment may be managed in accordance with a subscription model. The subscription model may use subscriptions to define the extent and limits on use of services provided by the deployment. The subscription services and limits may be enforced by management controllers of network devices of the deployment. The management controllers may operate independently from control planes and data planes of the network devices. If a plane is suspected of being compromised, the management controllers may take action to confirm the suspicions. If confirmed, then remedial activity may be initiated to address the compromised control plane.
1. A method of managing operation of a distributed system, the method comprising:
making, by a management controller of a first network device, an identification that a control plane of the first network device is suspected of being compromised;
identifying, by the management controller and based on the identification, a subscription for services provided by the first network device;
measuring, by the management controller and based on the subscription, a level of network service provided by the first network device that is governed by the subscription;
making a first determination regarding whether the level of network service is commensurate with the subscription;
in a first instance of the first determination where the level of network service is not commensurate with the subscription:
obtaining, by the management controller and based on the subscription, a network data package;
injecting, by the management controller, the network data package into a data plane of the first network device to initiate transmission of extra network data units based on the network data package to obtain an updated data plane; and
providing, by the first network device, network data unit processing services using the updated data plane.
2. The method of claim 1 , further comprising:
making a second determination regarding whether the level of network service provided by the updated data plane is commensurate with the subscription; and
in a first instance of the second determination where the level of network service provided by the updated data plane is commensurate with the subscription:
while providing the network data unit processing services:
dynamically adjusting, by the management controller, the network data package to retain the level of network service provided by the updated data plane.
3. The method of claim 1 , wherein the extra network data units are transmitted, at least in part, while the network data unit processing services are provided.
4. The method of claim 3 , wherein providing the network data unit processing services comprises:
obtaining, by the data plane, inbound network data units from other network devices of the distributed system; and
forwarding, by the data plane, the inbound network data units toward destinations.
5. The method of claim 4 , wherein the subscription specifies a network data units processing rate for the first network device, and content of the network data package is selected to consume a quantity of network data unit processing capacity of the data plane to prevent the data plane from processing the inbound network data units at a rate that exceeds the network data unit processing rate specified by the subscription.
6. The method of claim 4 , wherein the subscription specifies a network data units processing latency for the first network device, and content of the network data package is selected to consume a quantity of network data unit processing capacity of the data plane to prevent the data plane from processing the inbound network data units at a latency that is superior to the latency specified by the subscription.
7. The method of claim 1 , wherein the management controller comprises a data processing system, the control plane is hosted by computing resources of the first network device, and the data processing system operates independently from the computing resources.
8. The method of claim 7 , wherein the management controller is operably connected to the computing resources via a first management channel, and the first management channel being usable by the management controller to configure the computing resources.
9. The method of claim 8 , wherein the data plane is hosted by a special purposes hardware device operably connected to in-band links through which network traffic is obtained and forwarded on to other devices, and the management controller is operably connected to at least one other device via an out-of-band link.
10. The method of claim 9 , wherein the management controller is operably connected to the special purpose hardware device via a second management channel, and the second management channel being usable by the management controller to configure the special purpose hardware device.
11. The method of claim 10 , wherein the special purpose hardware device comprises a switch application specific integrated circuit adapted to forward network traffic.
12. The method of claim 1 , wherein the subscription is for a level of service to be provided by the first network device to a subscribing entity.
13. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause a network device to perform operations for managing operation of a distributed system, the operations comprising:
making, by a management controller of a first network device, an identification that a control plane of the first network device is suspected of being compromised;
identifying, by the management controller and based on the identification, a subscription for services provided by the first network device;
measuring, by the management controller and based on the subscription, a level of network service provided by the first network device that is governed by the subscription;
making a first determination regarding whether the level of network service is commensurate with the subscription;
in a first instance of the first determination where the level of network service is not commensurate with the subscription:
obtaining, by the management controller and based on the subscription, a network data package;
injecting, by the management controller, the network data package into a data plane of the first network device to initiate transmission of extra network data units based on the network data package to obtain an updated data plane; and
providing, by the first network device, network data unit processing services using the updated data plane.
14. The non-transitory machine-readable medium of claim 13 , wherein the operations further comprise
making a second determination regarding whether the level of network service provided by the updated data plane is commensurate with the subscription; and
in a first instance of the second determination where the level of network service provided by the updated data plane is commensurate with the subscription:
while providing the network data unit processing services:
dynamically adjusting, by the management controller, the network data package to retain the level of network service provided by the updated data plane.
15. The non-transitory machine-readable medium of claim 13 , wherein the extra network data units are transmitted, at least in part, while the network data unit processing services are provided.
16. The non-transitory machine-readable medium of claim 14 , wherein providing the network data unit processing services comprises:
obtaining, by the data plane, inbound network data units from other network devices of the distributed system; and
forwarding, by the data plane, the inbound network data units toward destinations.
17. The non-transitory machine-readable medium of claim 16 , wherein the subscription specifies a network data units processing rate for the first network device, and content of the network data package is selected to consume a quantity of network data unit processing capacity of the data plane to prevent the data plane from processing the inbound network data units at a rate that exceeds the network data unit processing rate specified by the subscription.
18. The non-transitory machine-readable medium of claim 16 , wherein the subscription specifies a network data units processing latency for the first network device, and content of the network data package is selected to consume a quantity of network data unit processing capacity of the data plane to prevent the data plane from processing the inbound network data units at a latency that is superior to the latency specified by the subscription.
19. A first network device, comprising:
a processor; and
a memory coupled to the processor to store instructions, which when executed by the processor, cause the first network device to perform operations for managing operation of a distributed system, the operations comprising:
making, by a management controller of a first network device, an identification that a control plane of the first network device is suspected of being compromised;
identifying, by the management controller and based on the identification, a subscription for services provided by the first network device;
measuring, by the management controller and based on the subscription, a level of network service provided by the first network device that is governed by the subscription;
making a first determination regarding whether the level of network service is commensurate with the subscription;
in a first instance of the first determination where the level of network service is not commensurate with the subscription:
obtaining, by the management controller and based on the subscription, a network data package;
injecting, by the management controller, the network data package into a data plane of the first network device to initiate transmission of extra network data units based on the network data package to obtain an updated data plane; and
providing, by the first network device, network data unit processing services using the updated data plane.
20. The first network device of claim 19 , wherein the operations further comprise
making a second determination regarding whether the level of network service provided by the updated data plane is commensurate with the subscription;
in a first instance of the second determination where the level of network service provided by the updated data plane is commensurate with the subscription:
while providing the network data unit processing services:
dynamically adjusting, by the management controller, the network data package to retain the level of network service provided by the updated data plane.