Distributed database configuration
Replicas are selected in a large distributed network, and the roles for these replicas are identified. In one example, a leader is selected from among candidate computing clusters. To make this selection, an activity monitor predicts or monitors the workload of one or more clients. Different activities of the workload are given corresponding weights. The delay in performing requested activities, modified by these weights is found, and the candidate leader with the lowest weighted delay is selected as the leader.
1. A computer-implemented method comprising:
generating, by data processing hardware, predicted workload data indicating a predicted weighted communication delay for each respective computing cluster of a plurality of computing clusters in performing one or more operations for a client application;
monitoring, by the data processing hardware, interactions between the client application and a distributed database executing the plurality of computing clusters, one of the plurality of computing clusters assigned a leader computing cluster role responsible for proposing operations for the plurality of computing clusters to perform on the distributed database based on the predicted weighted communication delay;
generating, by the data processing hardware and based on the interactions between the client application and the distributed database, historical workload data indicating a historical weighted communication delay for each respective computing cluster of the plurality of computing clusters in performing one or more operations for the client application;
determining, by the data processing hardware, that a different computing cluster of the plurality of computing clusters has less historical weighted communication delay than the leader computing cluster based on the historical workload data; and
based on determining that the different computing cluster of the plurality of computing clusters has less historical weighted communication delay than the leader computing cluster, assigning, by the data processing hardware, the different computing cluster the leader computing cluster role.
2. The computer-implemented method of claim 1 , wherein the distributed database comprises one or more voters that are configured to accept or deny the proposed operations of the one of the plurality of computing clusters assigned the leader computing cluster role.
3. The computer-implemented method of claim 2 , wherein the distributed database comprises one or more replicas that replicate at least a portion of the distributed database.
4. The computer-implemented method of claim 3 , wherein the operations further comprise, in response to assigning the different computing cluster the leader computing cluster role, assigning, by the data processing hardware, one or more new voters and one or more new replicas for the distributed database.
5. The computer-implemented method of claim 4 , wherein a quantity of the one or more new voters is equal in number to a quantity of the one or more voters.
6. The computer-implemented method of claim 4 , wherein a quantity of the one or more new replicas is equal in number to a quantity of the one or more replicas.
7. The computer-implemented method of claim 1 , wherein the operations further comprise, for each respective computing cluster:
determining, by the data processing hardware and based on the historical workload data, a communication delay for the respective computing cluster interacting with the client application;
identifying, by the data processing hardware, a frequency of the interactions; and
determining, by the data processing hardware, the historical weighted communication delay based on the communication delay and the frequency of the interactions.
8. The computer-implemented method of claim 1 , wherein the historical workload data comprises logs of the interactions between the client application and the distributed database.
9. A system comprising:
data processing hardware; and
memory hardware in communication with the data processing hardware, the memory hardware storing instructions that when executed on the data processing hardware cause the data processing hardware to:
generate predicted workload data indicating a predicted weighted communication delay for each respective computing cluster of a plurality of computing clusters in performing one or more operations for a client application;
monitor interactions between the client application and a distributed database executing the plurality of computing clusters, one of the plurality of computing clusters assigned a leader computing cluster role responsible for proposing operations for the plurality of computing clusters to perform on the distributed database based on the predicted weighted communication delay;
generate, based on the monitored interactions between the client application and the distributed database, historical workload data indicating a historical weighted communication delay for each respective computing cluster of the plurality of computing clusters in performing one or more operations for the client application;
determine that a different computing cluster of the plurality of computing clusters has less historical weighted communication delay than the leader computing cluster based on the historical workload data; and
based on determining that the different computing cluster of the plurality of computing clusters has less historical weighted communication delay than the leader computing cluster, assign the different computing cluster the leader computing cluster role.
10. The system of claim 9 , wherein the distributed database comprises one or more voters that are configured to accept or deny the proposed operations of the one of the plurality of computing clusters assigned the leader computing cluster role.
11. The system of claim 10 , wherein the distributed database comprises one or more replicas that replicate at least a portion of the distributed database.
12. The system of claim 11 , wherein the instructions further cause the data processing hardware to, in response to assigning the different computing cluster the leader computing cluster role, assign one or more new voters and one or more new replicas for the distributed database.
13. The system of claim 12 , wherein a quantity of the one or more new voters is equal in number to a quantity of the one or more voters.
14. The system of claim 12 , wherein a quantity of the one or more new replicas is equal in number to a quantity of the one or more replicas.
15. The system of claim 9 , wherein the instructions further cause the data processing hardware to, for each respective computing cluster:
determine, based on the historical workload data, a communication delay for the respective computing cluster interacting with the client application;
identify a frequency of the interactions; and
determine the historical weighted communication delay based on the communication delay and the frequency of the interactions.
16. The system of claim 9 , wherein the historical workload data comprises logs of the interactions between the client application and the distributed database.
17. A non-transitory computer-readable storage medium encoded with instructions that, when executed by one or more processors of a computing system, cause the one or more processors to:
generate predicted workload data indicating a predicted weighted communication delay for each respective computing cluster of a plurality of computing clusters in performing one or more operations for a client application;
monitor interactions between the client application and a distributed database executing the plurality of computing clusters, one of the plurality of computing clusters assigned a leader computing cluster role responsible for proposing operations for the plurality of computing clusters to perform on the distributed database based on the predicted weighted communication delay;
generate, based on the monitored interactions between the client application and the distributed database, historical workload data indicating a historical weighted communication delay for each respective computing cluster of the plurality of computing clusters in performing one or more operations for the client application;
determine that a different computing cluster of the plurality of computing clusters has less historical weighted communication delay than the leader computing cluster based on the historical workload data; and
responsive to determining that the different computing cluster of the plurality of computing clusters has less historical weighted communication delay than the leader computing cluster, assign the different computing cluster the leader computing cluster role.
18. The non-transitory computer-readable storage medium of claim 17 , wherein the distributed database comprises one or more voters that are configured to accept or deny the proposed operations of the one of the plurality of computing clusters assigned the leader computing cluster role.
19. The non-transitory computer-readable storage medium of claim 18 , wherein the distributed database comprises one or more replicas that replicate at least a portion of the distributed database.
20. The non-transitory computer-readable storage medium of claim 19 , wherein the instructions further cause the one or more processors to, in response to assigning the different computing cluster the leader computing cluster role, assign assigning one or more new voters and one or more new replicas for the distributed database.