Replication of hot data ranges in distributed transactional key-value stores
A hot server is identified from a plurality of servers based on one or more server metrics associated with the hot server. A hot data range stored by the hot server is identified based on one or more read density metrics. The hot data range comprises a range of data values with a higher volume of access requests compared to other data values stored by the hot server. The hot data range is replicated across a number of additional servers.
1 . A system comprising:
at least one hardware processor; and
at least one memory storing instructions that cause the at least one hardware processor to perform operations comprising:
identifying a hot server from a plurality of servers based on one or more server metrics associated with the hot server,
identifying a hot data range stored by the hot server based on one or more read density metrics associated with the hot data range, the hot data range comprising a range of data values with a higher volume of access requests relative to other data values stored by the hot server;
based on identifying the hot data range, replicating the hot data range across a number of additional servers;
determining, based on updated read density metrics associated with the hot data range, that the hot data range is no longer hot; and
based on determining that the hot data range is no longer hot, reducing a number of replicas of the hot data range, the reducing of the number of replicas comprising causing the hot data range to be deleted from one or more servers.
2 . The system of claim 1 , wherein the operations comprising:
prior to replicating the hot data range across the number of additional servers, verifying that the hot data range remains hot.
3 . The system of claim 2 , wherein:
the identifying of the hot range stored by the hot server is performed by the hot server; and
the verifying that the hot data range remains hot is performed by a data distributor.
4 . The system of claim 3 , wherein the data distributed distributor sends a request to the hot server to identify the hot data range in response to identifying the hot server.
5 . The system of claim 1 , wherein the number of additional servers is based on a fixed replication factor.
6 . The system of claim 1 , wherein operations comprise determining a replication factor based on a comparison of a current density of the hot data range to a maximum density constraint, wherein the current density of the hot data range is based on one or more of a read bandwidth of the hot data range and a read frequency of the hot data range, wherein the number of additional servers is based on the replication factor.
7 . The system of claim 1 , wherein the one or more server metrics comprise at least one of: read latency, storage queue size, durability lag, CPU usage, read bandwidth, read frequency, or query queue size.
8 . The system of claim 1 , wherein the one or more read density metrics comprise at least one of read bandwidth or read frequency.
9 . The system of claim 1 , wherein identifying the hot server from the plurality of servers comprises determining at least one server metric associated with the hot server satisfies a server metric threshold constraint.
10 . The system of claim 1 , wherein the identifying of the hot data range comprises:
determining at least one read density metric associated with the hot data range satisfies a read density metric threshold constraint; and
determining a size of the hot data range satisfies a size threshold constraint.
11 . The system of claim 1 , wherein the identifying of the hot data range comprises:
determining a ratio of read bandwidth to write bandwidth for the hot data range satisfies a threshold constraint; and
determining a size of the hot data range satisfies a size threshold constraint.
12 . The system of claim 1 , wherein determining that the hot range is no longer hot comprises determining at least one of the updated read density metrics does not satisfy a read density metric threshold constraint.
13 . The system of claim 1 , wherein the plurality of servers implements a key-value store, wherein the hot data range comprises a portion of values stored within the key-value store.
14 . A method comprising:
monitoring, by a data distributor comprising at least one processor, a plurality of server metrics of a plurality of servers implementing a key-value store;
identifying, by the data distributor, a hot server from the plurality of servers based on one or more server metrics of the hot server;
identifying, by the hot server, a hot data range stored by the hot server based on one or more read density metrics associated with the hot data range, the hot data range comprising a range of data values with a higher volume of access requests relative to other data values stored by the hot server;
based on identifying the hot data range, replicating the hot data range across a number of additional servers;
determining, based on updated read density metrics associated with the hot data range, that the hot data range is no longer hot; and
based on determining that the hot data range is no longer hot, reducing a number of replicas of the hot data range, the reducing of the number of replicas comprising causing the hot data range to be deleted from one or more servers.
15 . The method of claim 14 , comprising:
sending, by the data distributor, a request to the hot server to identify the hot data range in response to the data distributor identifying the hot server; and
prior to replicating the hot data range across the number of additional servers, verifying, by the data distributor, that the hot data range remains hot.
16 . The method of claim 14 , wherein:
the one or more server metrics comprise at least one of: read latency, storage queue size, durability lag, CPU usage, read bandwidth, read frequency, or query queue size; and
the one or more read density metrics comprise at least one of read bandwidth or read frequency.
17 . The method of claim 14 , wherein:
the identifying of the hot server from the plurality of servers comprises determining at least one server metric associated with the hot server satisfies a server metric threshold constraint; and
the identifying of the hot data range comprises:
determining at least one read density metric associated with the hot data range satisfies a read density metric threshold constraint; and
determining a size of the hot data range satisfies a size threshold constraint.
18 . The method of claim 14 , wherein determining that the hot range is no longer hot comprises determining at least one of the updated read density metrics does not satisfy a read density metric threshold constraint.
19 . Computer-storage media comprising instructions that, when executed by one or more processors of a machine, configure the machine to perform operations comprising:
monitoring a plurality of server metrics of a plurality of servers implementing a key-value store;
identifying a hot server from the plurality of servers based one or more server metrics of the hot server;
identifying a hot data range stored by the hot server based on one or more read density metrics associated with the hot data range, the hot data range comprising a range of data values with a higher volume of access requests relative to other data values stored by the hot server;
based on identifying the hot data range, replicating the hot data range across a number of additional servers;
determining, based on updated read density metrics associated with the hot data range, that the hot data range is no longer hot; and
based on determining that the hot data range is no longer hot, reducing a number of replicas of the hot data range, the reducing of the number of replicas comprising causing the hot data range to be deleted from one or more servers.
20 . The computer-storage media of claim 19 , wherein determining that the hot range is no longer hot comprises determining at least one of the updated read density metrics does not satisfy a read density metric threshold constraint.