Handling failed cluster members when replicating a database between clusters
Data integrity is maintained during failed communications between a member node of a primary cluster and a backup cluster by assigning an assisting member node to run an assisting process that transmits data entered into the member node to the backup cluster. In this way, a replicated database is maintained during a partial communication failure between the primary cluster and the backup cluster.
1. A method comprising:
sending, by a first node in a first cluster to a first node in a second cluster, at least a portion of a set of logs;
detecting a communication failure condition with respect to the sending of the set of logs;
responsive to detection of the communication failure condition, determining an assisting node which is in the first cluster; and
responsive to determination of the assisting node, sending, by the assisting node in the first cluster to the first node in the second cluster, at least a portion of the set of logs;
wherein:
the set of logs relate to physical database replication among shared disk database systems.
2. The method of claim 1 further comprising:
maintaining a configuration dataset for sending, by the first node in the first cluster to the first node in the second cluster, at least a portion of the set of logs.
3. The method of claim 1 wherein communication by the first node in the first cluster to the first node in the second cluster is restored when the communication failure condition with respect to the sending of the set of logs is resolved.
4. The method of claim 3 further comprising:
accepting a connection attempt by the first node in the first cluster to the first node in the second cluster.
5. The method of claim 1 wherein the physical database replication is an asynchronous process.
6. The method of claim 1 wherein the determining an assisting node which is in the first cluster comprises:
reassigning a network socket previously used for communication by the first node in the first cluster to the first node in the second cluster; and
changing an endpoint of the network socket from the first node to the assisting node.
7. The method of claim 6 wherein the determining an assisting node which is in the first cluster further comprises:
sending a request to the assisting node;
identifying the first node of the first cluster;
instructing the assisting node to create an assisting process that runs in parallel with a regular database process already running on the assisting node, the process including accessing the set of logs; and
forwarding the set of logs to a receiving node;
wherein:
the receiving node is a member node of the second cluster.
8. The method of claim 7 further comprising:
directing the assisting process to terminate.
9. The method of claim 1 wherein the set of logs includes all metadata that the first node of the first cluster generated during the communication failure.
10. A computer program product comprising a computer readable storage medium having stored thereon:
first program instructions programmed to send, by a first node in a first cluster to a first node in a second cluster, at least a portion of a set of logs;
second program instructions programmed to detect a communication failure condition with respect to the sending of the set of logs;
third program instructions programmed to, responsive to detection of the communication failure condition, determine an assisting node which is in the first cluster;
fourth program instructions programmed to, responsive to determination of the assisting node, send, by the assisting node in the first cluster to the first node in the second cluster, at least a portion of the set of logs;
wherein:
the set of logs relate to physical database replication among shared disk database systems.
11. The computer program product of claim 10 further comprising a computer readable storage medium having stored thereon:
fifth program instructions programmed to maintain a configuration dataset for sending, by the first node in the first cluster to the first node in the second cluster, at least a portion of the set of logs.
12. The computer program product of claim 10 wherein communication by the first node in the first cluster to the first node in the second cluster is restored when the communication failure condition with respect to the sending of the set of logs is resolved.
13. The computer program product of claim 12 further comprising a computer readable storage medium having stored thereon:
fifth program instructions programmed to accept a connection attempt by the first node in the first cluster to the first node in the second cluster.
14. The computer program product of claim 10 wherein the physical database replication is an asynchronous process.
15. The computer program product of claim 10 wherein the third program instructions programmed to determine an assisting node which is in the first cluster comprise:
program instructions to reassign a network socket previously used for communication by the first node in the first cluster to the first node in the second cluster; and
program instructions to change an endpoint of the network socket from the first node to the assisting node.
16. A computer system comprising:
a processor(s) set; and
a computer readable storage medium;
wherein:
the processor set is structured, located, connected and/or programmed to run program instructions stored on the computer readable storage medium; and
the program instructions include:
first program instructions programmed to send, by a first node in a first cluster to a first node in a second cluster, at least a portion of a set of logs;
second program instructions programmed to detect a communication failure condition with respect to the sending of the set of logs;
third program instructions programmed to, responsive to detection of the communication failure condition, determine an assisting node which is in the first cluster;
fourth program instructions programmed to, responsive to determination of the assisting node, send, by the assisting node in the first cluster to the first node in the second cluster, at least a portion of the set of logs;
wherein:
the set of logs relate to physical database replication among shared disk database systems.
17. The computer system of claim 16 further comprising a computer readable storage medium having stored thereon:
fifth program instructions programmed to maintain a configuration dataset for sending, by the first node in the first cluster to the first node in the second cluster, at least a portion of the set of logs.
18. The computer system of claim 16 wherein communication by the first node in the first cluster to the first node in the second cluster is restored when the communication failure condition with respect to the sending of the set of logs is resolved.
19. The computer system of claim 18 further comprising a computer readable storage medium having stored thereon:
fifth program instructions programmed to accept a connection attempt by the first node in the first cluster to the first node in the second cluster.
20. The computer system of claim 16 wherein the physical database replication is an asynchronous process.