Management services for distributed computing architectures using rolling changes
Management services for distributed computing architectures using rolling changes are provided herein. An example system includes clusters of nodes providing services and a plurality of management servers, each of the plurality of management servers including: at least a distributed coordination service for the clusters of nodes, the distributed coordination service being a datastore; and a constructor that manages allocation and life cycle deployments of the nodes of the clusters, the constructor further configured to manage topological changes to nodes of the clusters by implement rolling attribute changes for the nodes.
1. A system comprising:
clusters of nodes providing services; and
a plurality of management servers, each of the plurality of management servers comprising:
at least a distributed coordination service for the clusters of nodes, the distributed coordination service being a datastore;
a director that manages the distributed coordination service and promotes at least one of nodes of the clusters to being one of the plurality of management servers, wherein promoting comprises replicating the distributed coordination service; and
a constructor that manages allocation and life cycle deployments of one or more nodes of the clusters, the constructor further configured to:
initiate implementing rolling attribute changes for the clusters of nodes, the rolling attribute changes defining a processing order in which the nodes of the clusters are modified, the processing order identifying when two or more of the nodes are to be modified simultaneously or two or more of the nodes are to be modified sequentially;
determine at least one of the one or more nodes of the clusters for which the rolling attribute changes are initiated; and
control proxies to redirect traffic away from the at least one of the one or more nodes of the clusters for which the rolling attribute changes are initiated.
2. The system according to claim 1 , wherein a full cluster restart is performed when the implementing of the rolling attribute changes is initiated for each node in one of the clusters of nodes.
3. The system according to claim 1 , wherein when all the nodes of the cluster are to be modified simultaneously, the constructor performs a full cluster reboot.
4. The system according to claim 1 , wherein the processing order allows attribute changes to availability zones or deployment groups of the clusters of nodes.
5. The system according to claim 1 , wherein at least a portion of the clusters of nodes comprises master nodes, further wherein the processing order specifies that the master nodes have changes performed last, the master nodes coordinating additional nodes in a cluster.
6. The system according to claim 1 , wherein the processing order allows for an in-place change where at least node of a cluster is replaced by a new node.
7. The system according to claim 1 , wherein the constructor implements the rolling attribute changes upon a failure of any of a plurality of nodes in a cluster.
8. The system according to claim 1 , wherein the constructor implements the rolling attribute changes upon infrastructure changes or customer request.
9. The system according to claim 1 , wherein the system further comprises proxies, the constructor further controlling the proxies to redirect traffic away from any of:
a new node; and
a node that is about to be stopped or removed.
10. The system according to claim 1 , wherein the constructor temporarily boosts computing resources for a plurality of nodes of a cluster during instances when attribute changes are being performed.
11. A method, comprising:
providing clusters of nodes providing services;
providing a plurality of management servers, each of the plurality of management servers comprising at least a distributed coordination service for the clusters of nodes, the distributed coordination service being a datastore;
managing access to the distributed coordination service using a director;
promoting at least one of nodes of the clusters to being one of the plurality of management servers, wherein promoting comprises replicating the distributed coordination service; and
managing allocation and life cycle deployments of one or more nodes of the clusters using a constructor that is configured to:
initiate implementing rolling attribute changes for the clusters of nodes, the rolling attribute changes defining a processing order in which the nodes of the clusters are modified, the processing order identifying when two or more of the nodes are to be modified simultaneously or two or more of the nodes are to be modified sequentially;
determine at least one of the one or more nodes of the clusters for which the rolling attribute changes are initiated; and
control proxies to redirect traffic away from the at least one of the one or more nodes of the clusters for which the rolling attribute changes are initiated.
12. The method according to claim 11 , wherein a full cluster restart is performed when the implementing of the rolling attribute changes is initiated for each node in one of the clusters of nodes.
13. The method according to claim 11 , wherein when all the nodes of the cluster are to be modified simultaneously, the method comprises performing a full cluster reboot.
14. The method according to claim 11 , wherein the processing order allows attribute changes to availability zones or deployment groups of the clusters of nodes.
15. The method according to claim 11 , wherein at least a portion of the clusters of nodes comprises master nodes, further wherein the processing order specifies that the master nodes have attribute changes performed last, the master nodes coordinating additional nodes in a cluster.
16. The method according to claim 11 , wherein the constructor implements the rolling attribute changes upon any of a failure of any of a plurality of nodes in a cluster and infrastructure changes or customer request.
17. A system comprising:
clusters of nodes providing services; and
a plurality of management servers, each of the plurality of management servers comprising:
at least a distributed coordination service for the clusters of nodes, the distributed coordination service being a datastore; and
a constructor that manages allocation and life cycle deployments of nodes of the clusters, the constructor further configured to:
manage topological changes to the nodes of the clusters by initiating implementing rolling attribute changes for the nodes of the clusters according to a processing order that identifies when two or more of the nodes are to be modified simultaneously or two or more of the nodes are to be modified sequentially;
determine at least one of the nodes of the clusters for which the rolling attribute changes are initiated; and
control proxies to redirect traffic away from the at least one of the nodes of the clusters for which the rolling attribute changes are initiated.
18. The system according to claim 17 , wherein the system further comprises proxies, the constructor further controlling the proxies to redirect traffic away from any of:
a new node; and
a node that is about to be stopped or removed.
19. The system according to claim 17 , wherein the constructor temporarily boosts computing resources for a plurality of nodes of a cluster during instances when attribute changes are being performed.