Establishing quorums on an object-by-object basis within a management system
Techniques and mechanisms for establishing quorums on an object-by-object basis; e.g., by associating quorum schemes, quorum-eligibility groups and/or quorum rules with objects on an object-by-object basis. For example, a certain quorum scheme may be associated with one object in a system and a different quorum scheme may be associated with another object in the system. A management system may be configured to be part of a quorum-eligibility group of an object based on any of a variety of factors, including any of: the number of management systems in the system, the quorum rule to be employed; whether an instance of the object is stored in a same fault domain of at least one of the management systems that are members of the quorum scheme; and the desire to maintain high availability of the object in response to various failure scenarios.
1. For a system comprising a plurality of data storage objects residing in a plurality of fault domains, and a plurality of data storage management systems operative to manage the plurality of data storage objects, a first of the plurality of fault domains having a first data storage object and a second data storage object of the plurality of data storage objects resident therein, a method comprising acts of:
identifying two or more data storage management systems that are providing coordinated management of the first and the second data storage objects and determining that a communication failure has occurred between the identified two or more data storage management systems, wherein the identified two or more data storage management systems attempt to manage the first and second data storage objects independently after the communication failure has occurred;
providing a first quorum scheme having a first quorum-eligibility group of the plurality of data storage management systems eligible to manage one or more of the plurality of data storage objects;
providing a second quorum scheme having a second quorum-eligibility group, different from the first quorum-eligibility group, of the plurality of data storage management systems eligible to manage one or more of the plurality of data storage objects;
configuring at least a first of the plurality of data storage management systems to associate the first quorum scheme with the first data storage object; and
configuring the at least first data storage management system to associate the second quorum scheme with the second data storage object.
2. The method of claim 1 , wherein providing the first quorum scheme comprises defining the first quorum-eligibility group to exclude the first data storage management system.
3. The method of claim 1 , wherein providing the first quorum scheme comprises defining the first quorum-eligibility group to include the first data storage management system.
4. The method of claim 1 , wherein the at least first data storage management system has access and capability to manage the first data storage object, and
wherein providing the second quorum scheme comprises defining the first quorum-eligibility group to exclude the first data storage management system.
5. The method of claim 1 , further comprising:
determining a first quorum for the first data storage object based on the first quorum scheme; and determining a second quorum for the second data storage object based on the second quorum scheme.
6. The method of claim 5 , wherein the acts of determining the first quorum and determining the second quorum are performed in response to a communication failure in the system.
7. The method of claim 1 , wherein the first data storage object exists in each of a first subset of the plurality of fault domains, and
wherein providing the first quorum scheme comprises defining the first quorum scheme such that a majority of members of the first quorum-eligibility group reside in the first subset of the plurality of fault domains.
8. The method of claim 7 , wherein providing the first quorum scheme comprises configuring the first quorum-eligibility group for the first data storage object such that a same number of members of the first quorum-eligibility group reside in each fault domain of the first subset.
9. The method of claim 7 , wherein the first subset comprises only the first fault domain,
wherein providing the first quorum scheme comprises defining the first quorum-eligibility group to include an odd number of members.
10. The method of claim 7 , wherein the first quorum-eligibility group includes a non-managing quorum eligibility member.
11. The method of claim 7 , wherein the first fault domain is a first site, and the second fault domain is a second site separate from the first site.
12. The method of claim 1 , further comprising: enabling management of one or more of the plurality data storage objects by one or more of the plurality of data storage management systems in the event of a communication failure in the system while avoiding a split brain scenario.
13. A system comprising:
a plurality of data storage objects residing in a plurality of fault domains, a first of the plurality of fault domains having a first data storage object and a second data storage object of the plurality of data storage objects resident therein;
a plurality of data storage management systems operative to manage the plurality of data storage objects;
two or more data storage management systems operative to coordinate management of the first and second data storage objects and determine that a communication failure has occurred between the identified two or more data storage management systems, wherein the identified two or more data storage management systems attempt to manage the first and second data storage objects independently after the communication failure has occurred;
a first quorum scheme having a first quorum-eligibility group of the plurality of data storage management systems eligible to manage one or more of the plurality of data storage objects; and
a second quorum scheme having a second quorum-eligibility group, different from the first quorum-eligibility group, of the plurality of data storage management systems eligible to manage one or more of the plurality of data storage objects,
wherein at least a first of the plurality of data storage management systems is operative to: associate the first quorum scheme with the first data storage object; and associate the second quorum scheme with the second data storage object.
14. The system of claim 13 , wherein the at least first data storage management system resides in a second fault domain that does not include an instance of the first data storage object.
15. The system of claim 14 , wherein the first quorum scheme defines the first quorum-eligibility group to exclude the first data storage management system.
16. The system of claim 13 , wherein the at least first data storage management system has access and capability to manage the first data storage object, and
wherein the second quorum scheme defines the first quorum-eligibility group to exclude the first data storage management system.
17. The system of claim 13 , wherein the first data storage object exists in each of a first subset of the plurality of fault domains, and
wherein the first quorum scheme defines the first quorum scheme such that a majority of members of the first quorum-eligibility group reside in the first subset of the plurality of fault domains.
18. The system of claim 17 , wherein the first subset comprises only the first fault domain, and
wherein the first quorum scheme defines the first quorum-eligibility group to include an odd number of members.
19. The system of claim 17 , wherein the first fault domain is a first site, and the second fault domain is a second site separate from the first site.
20. A non-transitory computer-readable medium encoded with computer-executable instructions that, as a result of being executed by a computer, control the computer to perform, for a system comprising a plurality of data storage objects residing in a plurality of fault domains, and a plurality of data storage management systems operative to manage the plurality of data storage objects, a first of the plurality of fault domains having a first data storage object and a second data storage object of the plurality of data storage objects resident therein, a method comprising acts of:
identifying two or more data storage management systems that are providing coordinated management of the first and second data storage objects and determining that a communication failure has occurred between the identified two or more data storage management systems, wherein the identified two or more data storage management systems attempt to manage the first and the second data storage objects independently after the communication failure has occurred;
providing a first quorum scheme having a first quorum-eligibility group of the plurality of data storage management systems eligible to manage one or more of the plurality of data storage objects;
providing a second quorum scheme having a second quorum-eligibility group, different from the first quorum-eligibility group, of the plurality of data storage management systems eligible to manage one or more of the plurality of data storage objects;
configuring at least a first of the plurality of data storage management systems to associate the first quorum scheme with the first data storage object; and
configuring the at least first data storage management system to associate the second quorum scheme with the second data storage object.