Scalable distributed data store

1. A distributed database (DB) system, comprising:

a plurality of DB nodes, each DB node comprising a processor, a memory, a storage medium, and a network interface for communicating over a communication network;

one or more distributed DBs hosted by the plurality of DB nodes, each of the one or more distributed DBs comprising a plurality of DB partitions, wherein each DB partition is a process executed by a processor of a particular DB node representing either a master DB partition or a slave DB partition, wherein the master DB partition is configured to accept data requests and the slave DB partition is configured to synchronize with the master DB partition, wherein each different master DB partition resides on a different DB node, and wherein each slave DB partition resides on a DB node different than a DB node of a corresponding master DB partition;

at least one daemon process executed by at least one processor of at least one of the plurality of DB nodes, wherein the at least one daemon process:

accepts data requests and determines which DB partitions serve the requests;

upon a failure of a DB node of the plurality of DB nodes;

promotes at least one first slave DB partition hosted by a non-failed DB node to at least one first master DB partition, wherein the at least one first slave DB partition corresponds to at least one second master DB partition hosted by the failed DB node;

demotes the at least one second master DB partition to at least one second slave DB partition for a corresponding master DB partition and transitions the at least one second slave DB partition and slave DB partitions of the failed DB node to a new DB node; and

performs a rebalancing operation that re-distributes master DB partitions evenly across non-failed DB nodes with slave DB partitions residing on non-failed DB nodes different than the non-failed DB nodes of the corresponding master DB partitions.

2. The system of claim 1 , wherein when the at least one daemon process is executed by the at least one processor, a write and read performance of the one or more distributed DBs is maintained independent from a number of data objects stored in the one or more distributed DBs.

3. The system of claim 1 , further comprising an administrator tool configured to perform auto-sharding operations, wherein the auto-sharding operations comprise:

distributing data records among the at least one master DB partition and the at least one slave DB partition, wherein each of the data records is identified by a unique key string, and wherein the unique key string is utilized to determine a particular DB partition to which the data records are assigned and in which the data records are stored.

4. The system of claim 3 , wherein when the at least one daemon process is executed by the at least one processor, the at least one daemon process:

periodically generates a heartbeat message and sends the heartbeat message to the administrator tool, wherein the administrator tool stores the heartbeat message in a list of healthy daemons in a state store.

5. The system of claim 4 , wherein the administrator tool periodically compares the list of healthy daemons with a list of previously stored daemons and determines that a corresponding DB node is inoperable if the heartbeat message is not received within a predetermined period of time.

6. The system of claim 5 , wherein upon determining that the corresponding DB node is inoperable, the administrator tool starts a recovery operation.

7. A method comprising:

providing a plurality of DB nodes, each DB node comprising a processor, a memory, a storage medium, and a network interface for communicating over a communication network;

hosting one or more distributed DBs by the plurality of DB nodes, each of the one or more distributed DBs comprising a plurality of DB partitions, wherein each DB partition is a process executed by a processor of a particular DB node representing either a master DB partition or a slave DB partition, wherein the master DB partition is configured to accept data requests and the slave DB partition is configured to synchronize with the master DB partition, wherein each different master DB partition resides on a different DB node, and wherein each slave DB partition resides on a DB node different than a DB node of a corresponding master DB partition;

accepting, by at least one daemon process executed by at least one processor of at least one of the plurality of DB nodes, data requests;

determining, by the at least one daemon process, which DB partitions serve the requests; and

upon a failure of a DB node of the plurality of DB nodes, the at least one daemon process:

promoting at least one first slave DB partition hosted by a non-failed DB node to at least one first master DB partition, wherein the at least one first slave DB partition corresponds to at least one second master DB partition hosted by the failed DB node;

demoting the at least one second master DB partition to at least one second slave DB partition for a corresponding master DB partition and transitioning the at least one second slave DB partition and slave DB partitions of the failed DB node to a new DB node; and

performing a rebalancing operation that re-distributes master DB partitions evenly across non-failed DB nodes with slave DB partitions residing on non-failed DB nodes different than the non-failed DB nodes of the corresponding master DB partitions.

8. The method of claim 7 , further comprising:

maintaining a write and read performance of the one or more distributed DBs independent from a number of data objects stored in the one or more distributed DBs.

9. The method of claim 7 , further comprising:

performing auto-sharding operations, wherein the auto-sharding operations comprise:

distributing data records among the at least one master DB partition and the at least one slave DB partition,

wherein each of the data records is identified by a unique key string, and

wherein the unique key string is utilized to determine a particular DB partition to which the data records are assigned and in which the data records are stored.

10. The method of claim 9 , further comprising:

periodically generating a heartbeat message and sending the heartbeat message to an administrator tool, wherein the administrator tool stores the heartbeat message in a list of healthy daemons in a state store.

11. The method of claim 10 , wherein the administrator tool periodically compares the list of healthy daemons with a list of previously stored daemons and determines that a corresponding DB node is inoperable if the heartbeat message is not received within a predetermined period of time.

12. The method of claim 11 , further comprising:

upon determining that the corresponding DB node is inoperable, starting a recovery operation.

13. A computer program product comprising:

a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising computer readable program code configured to:

accept data requests and determine which DB partitions serve the requests;

upon a failure of a DB node of a plurality of DB nodes;

promote at least one first slave DB partition hosted by a non-failed DB node to at least one first master DB partition, wherein the at least one first slave DB partition corresponds to at least one second master DB partition hosted by the failed DB node;

demote the at least one second master DB partition to at least one second slave DB partition for a corresponding master DB partition and transition the at least one second slave DB partition and slave DB partitions of the failed DB node to a new DB node; and

perform a rebalancing operation that re-distributes master DB partitions evenly across non-failed DB nodes with slave DB partitions residing on non-failed DB nodes different than the non-failed DB nodes of the corresponding master DB partitions.

14. The computer program product of claim 13 , wherein the computer readable program code is further configured to:

maintain a write and read performance of one or more distributed DBs independent from a number of data objects stored in the one or more distributed DBs.

15. The computer program product of claim 14 , wherein the computer readable program code is configured to perform auto-sharding operations by an administrator tool, and wherein the auto-sharding operations comprise:

distributing data records among the at least one master DB partition and the at least one slave DB partition, wherein each of the data records is identified by a unique key string, and wherein the unique key string is utilized to determine a particular DB partition to which the data records are assigned and in which the data records are stored.

16. The computer program product of claim 15 , wherein the computer readable program code is further configured to:

periodically generate a heartbeat message and send the heartbeat message to the administrator tool, wherein the administrator tool stores the heartbeat message in a list of healthy daemons in a state store.

17. The computer program product of claim 16 , wherein the list of healthy daemons is periodically compared with a list of previously stored daemons,

wherein it is determined that a corresponding DB node is inoperable if the heartbeat message is not received within a predetermined period of time, and

wherein upon determining that the corresponding DB node is inoperable, a recovery operation is started by the administrator tool.