Method and system for providing distributed management in a networked virtualization environment
View Patent ↗A method for handling failure in a networked virtualization environment having distributed virtual machine management.
1. A method for handling failure in a networked virtualization environment having distributed virtual machine management, comprising:
identifying a failed management virtual machine instance of a plurality of management virtual machine instances in the networked virtualization environment,
wherein the networked virtualization environment comprises a plurality of nodes, where a node of the plurality of nodes comprises a hypervisor, one or more virtualization components and a management virtual machine instance that runs on top of the hypervisor,
wherein each management virtual machine instance of the plurality of management virtual machine instances services a subset of virtualization components in the networked virtualization environment, and
wherein a management virtual machine instance of the plurality of management virtual machine instances has access to a corresponding shard of a distributed database for the networked virtualization environment, the corresponding shard storing a subset of management data for virtualization components managed by the management virtual machine instance;
designating one or more active management virtual machine instances of the plurality of management virtual machine instances for replacing virtualization component management functionality of the failed management virtual machine instance,
wherein the one or more active management virtual machine instances replace virtualization component management functionality of the failed management virtual machine instance by each accessing its subset of management data in its corresponding shard of the distributed database, and
wherein the one or more active management virtual machine instances are configured to replace virtualization component management functionality of another management virtual machine instance whenever the other management virtual machine instance fails; and
distributing a workload of the failed management virtual machine instance amongst the one or more active management virtual machine instances in response to identifying the failed management virtual machine instance.
2. The method of claim 1 , wherein the subset of virtualization components in the networked virtualization environment being serviced by each management virtual machine instance comprises virtual machines.
3. The method of claim 2 , wherein the subset of management data for the networked virtualization environment being stored in the corresponding shard of a distributed database for each management virtual machine instance comprises virtual machine management data.
4. The method of claim 1 , further comprising:
determining whether the failed management virtual machine instance has recovered; and
reassigning the workload of the failed management virtual machine instance when the failed management virtual machine instance has recovered.
5. The method of claim 1 , wherein identifying the failed management virtual machine instance comprises:
identifying a state of the networked virtualization environment by the one or more active management virtual machine instances in a corresponding shard of the distributed database.
6. The method of claim 1 , wherein identifying the failed management virtual machine instance comprises:
utilizing pings between the failed management virtual machine instance and the one or more active management virtual machine instances.
7. The method of claim 1 , wherein the failed management virtual machine instance has itself failed rather than a node at which the failed management virtual machine instance resides.
8. The method of claim 7 , wherein distributing the workload of the failed management virtual machine instance amongst the one or more active management virtual machine instances comprises migrating virtual machine management functionality for one or more user virtual machines previously serviced by the failed management virtual machine instance to the one or more active management virtual machine instances.
9. The method of claim 1 , wherein the failed management virtual machine instance resides on a failing node of the networked virtualization environment.
10. The method of claim 9 , wherein distributing the workload of the failed management virtual machine instance amongst the one or more active management virtual machine instances comprises migrating one or more user virtual machines previously serviced by the failed management virtual machine instance to nodes at which the one or more active management virtual machine instances reside.
11. The method of claim 1 , wherein designating the one or more active management virtual machine instances for replacing the failing management virtual machine instance comprises:
electing an active management virtual machine instance as a leader management virtual machine instance; and
designating the one or more active management virtual machine instances for replacing the failing management virtual machine instance by the leader management virtual machine instance.
12. The method of claim 11 , wherein designating the one or more active management virtual machine instances is based on a current workload of the one or more active management virtual machine instances.
13. The method of claim 11 , wherein designating the one or more active management virtual machine instances is based on a workload of the failed management virtual machine instance.
14. A computer program product embodied on a non-transitory computer readable medium, the non-transitory computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute a method for handling failure in a networked virtualization environment having distributed virtual machine management, comprising:
identifying a failed management virtual machine instance of a plurality of management virtual machine instances in the networked virtualization environment,
wherein the networked virtualization environment comprises a plurality of nodes, where a node of the plurality of nodes comprises a hypervisor, one or more virtualization components and a management virtual machine instance that runs on top of the hypervisor,
wherein each management virtual machine instance of the plurality of management virtual machine instances services a subset of virtualization components in the networked virtualization environment, and
wherein a management virtual machine instance of the plurality of management virtual machine instances has access to a corresponding shard of a distributed database for the networked virtualization environment, the corresponding shard storing a subset of management data for virtualization components managed by the management virtual machine instance;
designating one or more active management virtual machine instances of the plurality of management virtual machine instances for replacing virtualization component management functionality of the failed management virtual machine instance,
wherein the one or more active management virtual machine instances replace virtualization component management functionality of the failed management virtual machine instance by each accessing its subset of management data in its corresponding shard of the distributed database, and
wherein the one or more active management virtual machine instances are configured to replace virtualization component management functionality of another management virtual machine instance whenever the other management virtual machine instance fails; and
distributing a workload of the failed management virtual machine instance amongst the one or more active management virtual machine instances in response to identifying the failed management virtual machine instance.
15. The computer program product of claim 14 , wherein the subset of virtualization components in the networked virtualization environment being serviced by each management virtual machine instance comprises virtual machines.
16. The computer program product of claim 15 , wherein the subset of management data for the networked virtualization environment being stored in the corresponding shard of a distributed database for each management virtual machine instance comprises virtual machine management data.
17. The computer program product of claim 14 , further comprising:
determining whether the failed management virtual machine instance has recovered; and
reassigning the workload of the failed management virtual machine instance when the failed management virtual machine instance has recovered.
18. The computer program product of claim 14 , wherein identifying the failed management virtual machine instance comprises:
identifying a state of the networked virtualization environment by the one or more active management virtual machine instances in a corresponding shard of the distributed database.
19. The computer program product of claim 14 , wherein identifying the failed management virtual machine instance comprises:
utilizing pings between the failed management virtual machine instance and the one or more active management virtual machine instances.
20. The computer program product of claim 14 , wherein the failed management virtual machine instance itself has failed rather than a node at which the failed management virtual machine instance resides.
21. The computer program product of claim 20 , wherein distributing the workload of the failed management virtual machine instance amongst the one or more active management virtual machine instances comprises migrating virtual machine management functionality for one or more user virtual machines previously serviced by the failed management virtual machine instance to the one or more active management virtual machine instances.
22. The computer program product of claim 14 , wherein the failed management virtual machine instance resides on a failing node of the networked virtualization environment.
23. The computer program product of claim 22 , wherein distributing the workload of the failed management virtual machine instance amongst the one or more active management virtual machine instances comprises migrating one or more user virtual machines previously serviced by the failed management virtual machine instance to nodes at which the one or more active management virtual machine instances reside.
24. The computer program product of claim 14 , wherein designating the one or more active management virtual machine instances for replacing the failing management virtual machine instance comprises:
electing an active management virtual machine instance as a leader management virtual machine instance; and
designating the one or more active management virtual machine instances for replacing the failing management virtual machine instance by the leader management virtual machine instance.
25. The computer program product of claim 24 , wherein designating the one or more active management virtual machine instances is based on a current workload of the one or more active management virtual machine instances.
26. The computer program product of claim 24 , wherein designating the one or more active management virtual machine instances is based on a workload of the failed management virtual machine instance.
27. A system for providing distributed virtual machine management to a networked virtualization environment, comprising:
a plurality of nodes, wherein each node of the plurality of nodes comprises a hypervisor and user virtual machines;
a management virtual machine instance on each of the plurality of nodes, wherein each management virtual machine instance runs above a corresponding hypervisor and services a subset of virtualization components in the networked virtualization environment;
a distributed database, wherein the distributed database is partitioned into shards corresponding to each of the management virtual machine instances, each shard comprising a subset of management data for the networked virtualization environment and each shard being accessible by its corresponding management virtual machine instance; and
wherein one or more active management virtual machine instances in the networked virtualization environment replaces virtual machine management functionality of a failed management virtual machine instance in the networked virtualization environment by each accessing its corresponding shard in the distributed database.