Intelligent rolling update of a cluster of servers via container orchestration

1. A method for automatically updating a plurality of brokers in a cluster of a distributed streaming platform within a container orchestration tool comprising:

querying, via an update manager executed by at least one processor, a StatefulSet that maintains each of a plurality of brokers to obtain metadata from a distributed streaming platform, the container orchestration tool being configured to orchestrate the distributed streaming platform and each of the plurality of brokers being configured to communicate via the distributed streaming platform;

using the obtained metadata, via the update manager, to identify a broker within the plurality of brokers that acts as a controlling broker;

computing, via the update manager, an order in which the plurality of brokers will be updated based on the obtained metadata, wherein the order specifies that the controlling broker will be updated last, wherein computing the order comprises:

identifying a first broker that stores a largest amount of data;

identifying a second broker that stores a second largest amount of data; and

computing the order such that a period of time between updates of the first broker and the second broker is maximized; and

performing, via the update manager, a rolling update of each of the plurality of brokers according to the computed order.

2. The method of claim 1 comprising, upon completion of the rolling update, checking, via the update manager, that each of the plurality of brokers is running.

3. The method of claim 1 , comprising:

checking, via the update manager, if a new version of the distributed streaming platform is available to download for the plurality of brokers; and

in response to determining that the new version is available, executing steps of the querying, the using, the computing, and the performing.

4. The method of claim 3 , wherein the checking comprises a public uniform resource locator (URL).

5. The method of claim 1 wherein performing the rolling update comprises:

terminating a beginning broker of the plurality of brokers according to the computed order;

updating existing metadata within the beginning broker to a newer version;

restarting the beginning broker; and

repeating steps of the terminating, the updating, and the restarting for each successive broker according to the computed order until each broker of the plurality of brokers has been updated.

6. The method of claim 1 , wherein the obtained metadata comprises:

an identity of each of the plurality of brokers;

the identity of the controlling broker; and

priority data for each of the plurality of brokers.

7. The method of claim 1 , wherein the order in which the plurality of brokers will be updated is computed based at least in part on amounts of data handled by each of the plurality of brokers.

8. The method of claim 1 , wherein the computed order in which the plurality of brokers will be updated is contained in a JSON object.

9. A method for automatically updating a plurality of brokers in a cluster of a distributed streaming platform within a container orchestration tool comprising:

obtaining metadata for a plurality of brokers within a container from a StatefulSet that maintains the plurality of brokers, the container orchestration tool being configured to orchestrate the distributed streaming platform and each of the plurality of brokers being configured to communicate via the distributed streaming platform;

sending, by the container executed by at least one processor, the metadata for a plurality of brokers within the container to a server, wherein the metadata is used to identify a broker within the plurality of brokers that acts as a controlling broker;

receiving, by the container, rolling updates for the plurality of brokers from the server, the updates being applied in an order computed using the obtained metadata, wherein the order specifies that the controlling broker will receive an update last, wherein the order further specifies that a period of time between updates of a first broker and a second broker is maximized, the first broker storing a largest amount of data and the second broker storing a second largest amount of data; and

applying, by the container, the rolling updates to the plurality of brokers according to the order.

10. The method of claim 9 comprising, upon completion of the rolling updates, receiving, by the container, a check that each of the plurality of brokers is running.

11. The method of claim 9 , wherein receiving the rolling updates comprises:

receiving, by the container, a signal to terminate a beginning broker of the plurality of brokers according to the computed order;

receiving, by the container, updates to existing metadata within the beginning broker;

receiving, by the container, a signal to restart the beginning broker; and

receiving, by the container, a signal to terminate, updates to metadata, and a signal to restart for each successive broker according to the computed order until each broker of the plurality of brokers has been updated.

12. The method of claim 9 , wherein the sent metadata comprises:

an identity of each of the plurality of brokers;

the identity of the controlling broker; and

priority data for each of the plurality of brokers.

13. The method of claim 9 , wherein the order in which the plurality of brokers will be updated is computed based at least in part on amounts of data handled by each of the plurality of brokers, via the server.

14. A system for automatically updating a plurality of brokers in a cluster of a distributed streaming platform within a container orchestration tool comprising:

a cluster of containers, each container including a plurality of brokers;

a server device comprising at least one processor; and

a non-transitory computer-readable medium comprising instructions stored thereon which, when executed by the server device, cause the at least one processor to execute a process operable to:

query, via an update manager executed by the at least one processor, a StatefulSet that maintains each of the plurality of brokers to obtain metadata from a distributed streaming platform, the container orchestration tool being configured to orchestrate the distributed streaming platform and each of the plurality of brokers being configured to communicate via the distributed streaming platform;

use the obtained metadata, via the update manager, to identify a broker within the plurality of brokers that acts as a controlling broker;

compute, via the update manager, an order in which the plurality of brokers will be updated based on the obtained metadata, wherein the order specifies that the controlling broker will be updated last, wherein computing the order comprises:

identifying a first broker that stores a largest amount of data;

identifying a second broker that stores a second largest amount of data; and

computing the order such that a period of time between updates of the first broker and the second broker is maximized; and

perform, via the update manager, a rolling update of each of the plurality of brokers according to the computed order.

15. The system of claim 14 , wherein the non-transitory computer-readable medium comprises instructions that, when executed by the at least one processor, cause the at least one processor to, upon completion of the rolling update, check that each of the plurality of brokers is running.

16. The system of claim 14 , wherein the non-transitory computer-readable medium comprises further instructions that, when executed by the at least one processor, cause the at least one processor to:

check if a new version of the distributed streaming platform is available to download for the plurality of brokers; and

in response to determining that the new version is available, execute steps of the querying, the using, the computing, and the performing.

17. The system of claim 14 , wherein performing the rolling update comprises:

terminating a beginning broker of the plurality of brokers according to the computed order;

updating existing metadata within the beginning broker to a newer version;

restarting the beginning broker; and

repeating steps of the terminating, the updating, and the restarting for each successive broker according to the computed order until each broker of the plurality of brokers has been updated.

18. The system of claim 14 , wherein the obtained metadata comprises:

an identity of each of the plurality of brokers;

the identity of the controlling broker; and

priority data for each of the plurality of brokers.

19. The system of claim 14 , wherein the order in which the plurality of brokers will be updated is computed based at least in part on amounts of data handled by each of the plurality of brokers via the server device.