Ensuring globally consistent transactions
The present technology proposes techniques for ensuring globally consistent transactions. This technology may allow distributed systems to ensure the causal order of read and write transactions across different partitions of a distributed database. By assigning causally generated timestamps to the transactions based on one or more globally coherent time services, the timestamps can be used to preserve and represent the causal order of the transactions in the distributed system. In this regard, certain transactions may wait for a period of time after choosing a timestamp in order to delay the start of any second transaction that might depend on it. The wait may ensure that the effects of the first transaction are not made visible until its timestamp is guaranteed to be in the past. This may ensure that a consistent snapshot of the distributed database can be determined for any past timestamp.
1. A distributed system having one or more processors and one or more storage devices, the distributed system comprising:
one or more time masters;
a time synchronization daemon in communication with the one or more time masters, the time synchronization daemon configured to synchronize local time by:
periodically querying the one or more time masters;
tracking round-trip delays associated with querying the one or more time masters, each round-trip delay indicating an amount of time between sending a query to one of the time masters and receiving a response to the query from the one of the time masters; and
calculating a current-time interval based on the round-trip delays; and
a library interface in communication with the one or more time masters and the time synchronization daemon, the library interface configured to provide applications with the current-time interval that contains a current true time for causally timestamping transactions occurring across different geographic locations.
2. The distributed system of claim 1 , wherein the one or more time masters are deployed at multiple host machines.
3. The distributed system of claim 1 , wherein the time synchronization daemon is further configured to calculate synchronization offset intervals relative to each of the one or more time masters based on tracking the round-trip delays.
4. The distributed system of claim 1 , wherein the time synchronization daemon is further configured to:
receive, from the one or more time masters, responses to periodic time queries, wherein the received responses include response timestamps generated by the one or more time masters;
calculate, for each of the received responses, a time offset interval, wherein a length of time duration of the time offset interval represents a level of uncertainty associated with the query to and from a particular time master; and
determine an agreement validated offset interval consistent among a selected group of the one or more time masters based on the calculated time offset intervals.
5. The distributed system of claim 4 , wherein the time synchronization daemon is further configured to:
identify the time masters outside of the selected group as producing inaccurate timestamps; and
reject time offset intervals from the identified time masters.
6. The distributed system of claim 4 , wherein the time synchronization daemon is further configured to:
identify the time masters within the selected groups as producing accurate timestamps; and
validate time offset intervals from the identified time masters.
7. The distributed system of claim 6 , wherein the library interface is further configured to transmit the validated time offset intervals to the applications.
8. A method for synchronizing one or more time masters of a distributed system, the method comprising:
synchronizing, by a time synchronization daemon, local time by:
periodically querying the one or more time masters;
tracking round-trip delays associated with querying the one or more time masters, each round-trip delay indicating an amount of time between sending a query to one of the time masters and receiving a response to the query from the one of the time masters; and
calculating a current-time interval based on the round-trip delays; and
providing, by a library interface, applications with the current-time interval that contains a current true time for causally timestamping transactions occurring across different geographic locations.
9. The method of claim 8 , wherein the one or more time masters are deployed at multiple host machines.
10. The method of claim 8 , further comprising calculating, by the time synchronization daemon, synchronization offset intervals relative to each of the one or more time masters based on tracking the round-trip delays.
11. The method of claim 8 , further comprising:
receiving, by the time synchronization daemon, from the one or more time masters, responses to periodic time queries, wherein the received responses include response timestamps generated by the one or more time masters;
calculating, by the time synchronization daemon, for each of the received responses, a time offset interval, wherein a length of time duration of the time offset interval represents a level of uncertainty associated with the query to and from a particular time master; and
determining, by the time synchronization daemon, an agreement validated offset interval consistent among a selected group of the one or more time masters based on the calculated time offset intervals.
12. The method of claim 11 , further comprising:
identifying, by the time synchronization daemon, the time masters outside of the selected group as producing inaccurate timestamps; and
rejecting, by the time synchronization daemon, time offset intervals from the identified time masters.
13. The method of claim 11 , further comprising:
identifying, by the time synchronization daemon, the time masters within the selected groups as producing accurate timestamps; and
validating, by the time synchronization daemon, time offset intervals from the identified time masters.
14. The method of claim 13 , further comprising transmitting, by the library interface, the validated time offset intervals to the applications.
15. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform a method for synchronizing one or more time masters of a distributed system, the method comprising:
synchronizing, by a time synchronization daemon, local time by:
periodically querying the one or more time masters;
tracking round-trip delays associated with querying the one or more time masters, each round-trip delay indicating an amount of time between sending a query to one of the time masters and receiving a response to the query from the one of the time masters; and
calculating a current-time interval based on the round-trip delays; and
providing, by a library interface, applications with the current-time interval that contains a current true time for causally timestamping transactions occurring across different geographic locations.
16. The non-transitory computer-readable medium of claim 15 , wherein the method further comprises calculating, by the time synchronization daemon, synchronization offset intervals relative to each of the one or more time masters based on tracking the round-trip delays.
17. The non-transitory computer-readable medium of claim 15 , wherein the method further comprises:
receiving, by the time synchronization daemon, from the one or more time masters, responses to periodic time queries, wherein the received responses include response timestamps generated by the one or more time masters;
calculating, by the time synchronization daemon, for each of the received responses, a time offset interval, wherein a length of time duration of the time offset interval represents a level of uncertainty associated with the query to and from a particular time master; and
determining, by the time synchronization daemon, an agreement validated offset interval consistent among a selected group of the one or more time masters based on the calculated time offset intervals.
18. The non-transitory computer-readable medium of claim 17 , wherein the method further comprises:
identifying, by the time synchronization daemon, the time masters outside of the selected group as producing inaccurate timestamps; and
rejecting, by the time synchronization daemon, time offset intervals from the identified time masters.
19. The non-transitory computer-readable medium of claim 17 , wherein the method further comprises:
identifying, by the time synchronization daemon, the time masters within the selected groups as producing accurate timestamps; and
validating, by the time synchronization daemon, time offset intervals from the identified time masters.
20. The non-transitory computer-readable medium of claim 19 , wherein the method further comprises transmitting, by the library interface, the validated time offset intervals to the applications.