Multi-class heterogeneous clients in a clustered filesystem
View Patent ↗A cluster of computer system nodes connected by a storage area network include two classes of nodes. The first class of nodes can act as clients or servers, while the other nodes can only be clients. The client-only nodes require much less functionality and can be more easily supported by different operating systems. To minimize the amount of data transmitted during normal operation, the server responsible for maintaining a cluster configuration database repeatedly multicasts the IP address, its incarnation number and the most recent database generation number. Each node stores this information and when a change is detected, each node can request an update of the data needed by that node. A client-only node uses the IP address of the server to connect to the server, to download the information from the cluster database required by the client-only node and to upload local disk connectivity information.
1. A method for operating a cluster file system for a cluster connected by a storage area network, comprising:
at a client node having one or more processors and a local memory:
receiving, from a cluster database server maintaining a cluster configuration database, a periodic multicast message, wherein the periodic multicast message includes a cluster database server address, an incarnation number of the cluster database server, and a database generation identifier for the cluster configuration database;
receiving, from the cluster database server, cluster configuration information concerning a respective cluster maintained by the cluster database server;
monitoring changes in the cluster database server address, the incarnation number, and the database generation identifier included in the periodic multicast message over time;
upon detecting a change in at least one of the multicast cluster database server address, the incarnation number, and the database generation contained within the periodic multicast message over time, requesting updated cluster configuration information from the cluster database server;
based on the updated cluster configuration information obtained from the cluster database server, updating a locally stored copy of the cluster configuration information;
reconfiguring a kernel of the client node based on the updated cluster configuration information;
establishing a communication link with the cluster database server in accordance with the cluster database server address included in the periodic multicast message;
requesting from the cluster database server information needed to join the respective cluster, the information including an address for at least one switch in the storage area network;
obtaining from the at least one switch at least one port address of at least one port connected to at least one host bus adapter of the client node;
sending the at least one port address and an address of the at least one host bus adapter from the candidate node to the cluster database server to update the cluster configuration database concerning the respective cluster maintained by the cluster database server;
requesting membership in the respective cluster by the client node;
obtaining the cluster configuration data in response to requesting membership in the respective cluster; and
configuring kernel components of the client node to enable access to the storage area network by the client node upon acceptance as a member of the respective cluster, wherein the client node enters a reconfigure state upon receiving the cluster configuration data, and while in the cluster configuration state, the client node creates and accepts connections to and from other nodes in the respective cluster, and mounts and accesses file systems on shared storage devices in the storage area network.
2. The method of claim 1 , wherein the client node exits a bootstrap state after establishing the communication link to the cluster database server, and returns to the bootstrap state when a socket error is received in response to the request for information needed to join the respective cluster.
3. The method of claim 2 , wherein the client node exits a bootstrap state after establishing the communication link to the cluster database server, and transitions to a stuck state when other types of errors are received in response to the request for information needed to join the respective cluster, wherein the client node transitions back to the bootstrap state after a timeout.
4. The method of claim 1 , further comprising providing authentication information, at the client node, upon requesting membership in the respective cluster.
5. A non-transitory computer-readable medium having instructions stored thereon, the instructions, when executed by one or more processors, cause the processors to perform operations comprising:
at a client node of a cluster connected by a storage area network:
receiving, from a cluster database server maintaining a cluster configuration database, a periodic multicast message, wherein the periodic multicast message includes a cluster database server address, an incarnation number of the cluster database server, and a database generation identifier for the cluster configuration database;
receiving, from the cluster database server, cluster configuration information concerning a respective cluster maintained by the cluster database server;
monitoring changes in in at least one of the cluster database server address, the incarnation number, and the database generation identifier included in the periodic multicast message over time;
upon detecting a change in at least one of the multicast cluster database server address, the incarnation number, and the database generation, requesting updated cluster configuration information from the cluster database server;
based on the updated cluster configuration information obtained from the cluster database server, updating a locally stored copy of the cluster configuration information;
reconfiguring a kernel of the client node based on the updated cluster configuration information;
establishing a communication link with the cluster database server in accordance with the cluster database server address included in the periodic multicast message;
requesting from the cluster database server information needed to join the respective cluster, the information including an address for at least one switch in the storage area network;
obtaining from the at least one switch at least one port address of at least one port connected to at least one host bus adapter of the client node;
sending the at least one port address and an address of the at least one host bus adapter from the candidate node to the cluster database server to update the cluster configuration database concerning the respective cluster maintained by the cluster database server;
requesting membership in the respective cluster by the client node;
obtaining the cluster configuration data in response to requesting membership in the respective cluster; and
configuring kernel components of the client node to enable access to the storage area network by the client node upon acceptance as a member of the respective cluster, wherein the client node enters a reconfigure state upon receiving the cluster configuration data, and while in the cluster configuration state, the client node creates and accepts connections to and from other nodes in the respective cluster, and mounts and accesses file systems on shared storage devices in the storage area network.
6. The computer-readable medium of claim 5 , wherein the client node exits a bootstrap state after establishing the communication link to the cluster database server, and returns to the bootstrap state if a socket error is received in response to the request for information needed to join the respective cluster.
7. The computer-readable medium of claim 6 , wherein the client node exits a bootstrap state after establishing the communication link to the cluster database server, and or transitions to a stuck state when other types of errors are received in response to the request for information needed to join the respective cluster, wherein the client node transitions back to the bootstrap state after a timeout.
8. The computer-readable medium of claim 5 , wherein the operations further comprising providing authentication information upon requesting membership in the respective cluster.
9. A system, comprising:
one or more processors; and
memory including instructions executable by the one or more processors to cause the processors to perform operations comprising:
at a client node of a cluster connected by a storage area network:
receiving, from a cluster database server maintaining a cluster configuration database, a periodic multicast message, wherein the periodic multicast message includes a cluster database server address, an incarnation number of the cluster database server, and a database generation identifier for the cluster configuration database;
receiving, from the cluster database server, cluster configuration information concerning a respective cluster maintained by the cluster database server;
monitoring changes in at least one of the cluster database server address, the incarnation number, and the database generation identifier included in the periodic multicast message;
upon detecting a change in at least one of the multicast cluster database server address, the incarnation number, and the database generation contained within the periodic multicast message over time, requesting updated cluster configuration information from the cluster database server;
based on the updated cluster configuration information obtained from the cluster database server, updating a locally stored copy of the cluster configuration information;
reconfiguring a kernel of the client node based on the updated cluster configuration information;
establishing a communication link with the cluster database server in accordance with the cluster database server address included in the periodic multicast message;
requesting from the cluster database server information needed to join the respective cluster, the information including an address for at least one switch in the storage area network;
obtaining from the at least one switch at least one port address of at least one port connected to at least one host bus adapter of the client node;
sending the at least one port address and an address of the at least one host bus adapter from the candidate node to the cluster database server to update the cluster configuration database concerning the respective cluster maintained by the cluster database server;
requesting membership in the respective cluster by the client node;
obtaining the cluster configuration data in response to requesting membership in the respective cluster; and
configuring kernel components of the client node to enable access to the storage area network by the client node upon acceptance as a member of the respective cluster, wherein the client node enters a reconfigure state upon receiving the cluster configuration data, and while in the cluster configuration state, the client node creates and accepts connections to and from other nodes in the respective cluster, and mounts and accesses file systems on shared storage devices in the storage area network.
10. The system of claim 9 , wherein the client node exits a bootstrap state after establishing the communication link to the cluster database server, and returns to the bootstrap state if a socket error is received in response to the request for information needed to join the respective cluster.
11. The system of claim 10 , wherein the client node exits a bootstrap state after establishing the communication link to the cluster database server, and transitions to a stuck state when other types of errors are received in response to the request for information needed to join the respective cluster, wherein the client node transitions back to the bootstrap state after a timeout.
12. The system of claim 9 , wherein the operations further comprise providing authentication information at the client node upon requesting membership in the respective cluster.