IP Library › Granted Patent US 12681711
Granted Patent B1
US 12681711 · App. 18/539,056 · Granted Jul 14, 2026

Autonomic control plane scaling in a container orchestration cluster

Inventors: Prateek Gogia (Leander, TX); Nathan John Herz (Seattle, WA)
Assignee: Amazon Technologies, Inc.
G06F8/61G06F9/45558
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12681711
App. No.
18/539,056
Granted
Jul 14, 2026
Kind
B1
Abstract

Computer-implemented techniques for autonomic control plane scaling in a container orchestration cluster. The techniques include a method that involves managing a container orchestration cluster's control plane. When a control plane instance is underutilized, it handles control plane requests and monitors its resource usage. If it becomes overutilized (hot state), it identifies another instance with spare resources (cold state) and redirects control plane traffic to it to maintain performance and availability.

Claims (84)

1 . A method comprising:

at a control plane instance in a control plane of a container orchestration cluster:

while the control plane instance is in a cold compute resource utilization state, receiving and processing control plane request traffic, and monitoring utilization of one or more compute resources of the control plane instance;

wherein the control plane comprises a plurality of control plane instances, the plurality of control instances comprising the control plane instance;

detecting that the control plane instance is in a hot compute resource utilization state;

determining from a distributed key-value store that each other control plane instance of the plurality of control plane instances is also in a hot compute resource utilization state;

causing a new control plane instance to be added to the control plane; and

after determining from the distributed key-value store that the new control plane instance has been added to the control plane and is ready to receive and process control plane request traffic, redirecting network traffic comprising control plane request traffic from the control plane instance to the new control plane instance.

2 . The method of claim 1 , further comprising:

at the new control plane instance:

while the new control plane instance is in a cold compute resource utilization state, receiving and processing control plane request traffic, and monitoring utilization of one or more compute resources of the new control plane instance;

detecting that the new control plane instance is in a hot compute resource utilization state;

determining from the distributed key-value store that each control plane instance of the plurality of control plane instances is also in a hot compute resource utilization state; and

while the new control plane instance is in a hot compute resource utilization state, processing control plane request traffic received from the control plane instance.

3 . The method of claim 1 , further comprising:

at the new control plane instance:

while the new control plane instance is in a cold compute resource utilization state, receiving and processing control plane request traffic, and monitoring utilization of one or more compute resources of the new control plane instance;

detecting that the new control plane instance is in a hot compute resource utilization state;

determining from the distributed key-value store that the control plane instance is in a cold compute resource utilization state; and

while the new control plane instance is in a hot compute resource utilization state, redirecting network traffic comprising control plane request traffic from the new control plane instance to the control plane instance.

4 . A method comprising:

at a first control plane instance in a control plane of a container orchestration cluster: while the first control plane instance is in a cold compute resource utilization state, receiving and processing control plane request traffic, and monitoring utilization of one or more compute resources of the first control plane instance;

detecting that the first control plane instance is in a hot compute resource utilization state;

causing a second control plane instance to be added to the control plane:

determining, at the first control plane instance, that the second control plane instance in the control plane that is in a cold compute resource utilization state, based on data from a distributed key-value store indicating that the second control plane instance has been added to the control plane and is ready to receive and process control plane request traffic; and

while the first control plane instance is in a hot compute resource utilization state, redirecting network traffic comprising control plane request traffic from the first control plane instance to the second control plane instance.

5 . The method of claim 4 , wherein determining, at the first control plane instance, that the second control plane instance is in a cold compute resource utilization state is based on determining, at the first control plane instance, from the distributed key-value store that the second control plane instance is not in a lock state.

6 . The method of claim 4 , further comprising:

at the second control plane instance:

while the second control place instance is in a cold compute resource utilization state, receiving and processing control plane request traffic, and monitoring utilization of one or more compute resources of the second control plane instance;

detecting that the second control plane instance is in a hot compute resource utilization state;

locking the second control plane instance in the distributed key-value store;

determining from the distributed key-value store that each control plane instance of a plurality of control plane instances is also in a hot compute resource utilization state; and

while the second control plane instance is in a hot compute resource utilization state, processing control plane request traffic received from the first control plane instance.

7 . The method of claim 4 , further comprising:

at the second control plane instance:

while the second control plane instance is in a cold compute resource utilization state, receiving and processing control plane request traffic, and monitoring utilization of one or more compute resources of the second control plane instance;

detecting that the second control plane instance is in a hot compute resource utilization state;

determining from the distributed key-value store that the first control plane instance is in a cold compute resource utilization state; and

while the second control plane instance is in a hot compute resource utilization state, redirecting network traffic comprising control plane request traffic from the second control plane instance to the first control plane instance.

8 . The method of claim 4 , further comprising:

at the first control plane instance:

detecting that the first control plane instance is in a hotter compute resource utilization state; and

while the first control plane instance is in a hotter compute resource utilization state, redirecting an increased amount of network traffic comprising control plane request traffic from the first control plane instance to the second control plane instance.

9 . The method of claim 4 , further comprising:

at the first control plane instance:

detecting that the first control plane instance in a cold compute resource utilization state; and

while the first control plane instance is in a cold compute resource utilization state, ceasing redirecting of network traffic comprising control plane request traffic from the first control plane instance to the second control plane instance.

10 . The method of claim 4 , wherein the first control plane instance comprises a virtual machine instance; wherein monitoring, at the first control plane instance, utilization of one or more compute resources of the first control plane instance comprises monitoring, at the first control plane instance, a processor utilization of the virtual machine instance; and wherein detecting, at the first control plane instance, that the first control plane instance is in a hot compute resource utilization state is based on detecting, at the first control plane instance, that the processor utilization of the virtual machine instance exceeds a threshold.

11 . The method of claim 4 , wherein the first control plane instance comprises a virtual machine instance; wherein monitoring, at the first control plane instance, utilization of one or more compute resources of the first control plane instance comprises monitoring, at the first control plane instance, a memory utilization of the virtual machine instance; and wherein detecting, at the first control plane instance, that the first control plane instance is in a hot compute resource utilization state is based on detecting, at the first control plane instance, that the memory utilization of the virtual machine instance exceeds a threshold.

12 . The method of claim 4 , wherein redirecting network traffic comprising control plane request traffic from the first control plane instance to the second control plane instance comprises:

at the first control plane instance:

forwarding internet protocol (IP) packets to the second control plane instance.

13 . The method of claim 4 , wherein redirecting network traffic comprising control plane request traffic from the first control plane instance to the second control plane instance comprises:

at the first control plane instance:

forwarding internet protocol (IP) packets to the second control plane instance with network address translation (NAT).

14 . The method of claim 4 , wherein redirecting network traffic comprising control plane request traffic from the first control plane instance to the second control plane instance comprises:

at the first control plane instance:

proxying internet protocol (IP) packets received from one or more clients of the first control plane instance to the second control plane instance.

15 . The method of claim 4 , wherein monitoring, at the first control plane instance, utilization of one or more compute resources of the first control plane instance comprises monitoring, at the first control plane instance, a control plane request rate; and wherein detecting, at the first control plane instance, that the first control plane instance is in a hot compute resource utilization state is based on detecting, at the first control plane instance, that a control plane request rate exceeds a threshold.

16 . The method of claim 4 , further comprising:

in response to detecting, at the first control plane instance, that the first control plane instance is in a hot compute resource utilization state, setting the first control plane instance in a lock state in the distributed key-value store; and

at a second control plane instance in the control plane:

while the second control plane instance is in a cold compute resource utilization state, receiving and processing control plane request traffic, and monitoring utilization of one or more compute resources of the second control plane instance;

wherein the control plane comprises a plurality of control plane instances, the plurality of control instances comprising the first control plane instance and the second control instance;

detecting that the second control plane instance is in a hot compute resource utilization state;

determining from the distributed key-value store that first control plane instance is also in a hot compute resource utilization state;

causing a third control plane instance to be added to the control plane; and

after determining from the distributed key-value store that the third control plane instance has been added to the control plane and is ready to receive and process control plane request traffic, redirecting network traffic comprising control plane request traffic from the second control plane instance to the third control plane instance.

17 . A system comprising:

one or more processors of a first control plane instance in a control plane of a container orchestration cluster; and

a memory of the first control plane instance storing instructions to be processed by the one or more processors of the first control plane instance, the instructions for:

while the first control plane instance is in a cold compute resource utilization state, receiving and processing control plane request traffic, and monitoring utilization of one or more compute resources of the first control plane instance;

detecting that the first control plane instance is in a hot compute resource utilization state;

causing a second control plane instance to be added to the control plane:

determining that the second control plane instance in the control plane that is in a cold compute resource utilization state based on data from a distributed key-value store indicating that the second control plane instance has been added to the control plane and is ready to receive and process control plane request traffic; and

while the first control plane instance is in a hot compute resource utilization state, redirecting network traffic comprising control plane request traffic to the second control plane instance.

18 . The system of claim 17 , further comprising:

one or more processors of a second control plane instance in the control plane; and

a memory of the second control plane instance storing instructions to be processed by the one or more processors of the second control plane instance, the instructions for:

while the second control place instance is in a cold compute resource utilization state, receiving and processing control plane request traffic, and monitoring utilization of one or more compute resources of the second control plane instance;

detecting that the second control plane instance is in a hot compute resource utilization state;

determining from the distributed key-value store that each control plane instance of a plurality of control plane instances is also in a hot compute resource utilization state; and

while the second control plane instance is in a hot compute resource utilization state, processing control plane request traffic received from the first control plane instance.