Transaction management for monotonic write consistency in a distributed storage system
Systems and methods are provided for managing the order of data written to a transaction log in a distributed storage system. In a system with multiple nodes, if sequencers are naively generated without taking into consideration inconsistencies among the different nodes generating the sequencers, then the sequencers may not increase for each data transaction. To alleviate this problem, the node committing the transaction to the transaction log may perform a consistent read and verifies that the sequencer advances. If the sequencer does not advance, the node can perform a context-dependent operation such as adjusting the sequencer, acknowledging the write without committing it to the transaction log, or rejecting the write altogether.
1. A distributed storage system comprising:
a plurality of front-end nodes, wherein a front-end node of the plurality of front-end nodes comprises a processor, memory, and a system clock; and
a storage service comprising a transaction log, a coordination node, and a plurality of persistent storage nodes, wherein the transaction log stores a record comprising a first sequence number and first data associated with a key;
wherein a first front-end node of the plurality of front-end nodes is configured to at least:
receive a data write request comprising second data associated with the key, wherein the second data is different from the first data;
generate a second sequence number based at least partly on the system clock; and
send the key, the second data, and the second sequence number to the coordination node; and
wherein the coordination node is configured to:
obtain the first sequence number from the transaction log based at least partly on the key;
determine that the second sequence number is less than the first sequence number;
determine that the second data is different from the first data; and
reject the data write request based at least partly on the second sequence number being less than the first sequence number and the second data being different from the first data.
2. The distributed storage system of claim 1 , wherein the transaction log further stores a second record comprising a third sequence number and third data associated with a second key, and wherein the first front-end node is further configured to at least:
receive a second data write request comprising fourth data associated with the second key, wherein the fourth data is different from the third data;
generate a fourth sequence number based at least partly on the system clock;
obtain the third sequence number from the transaction log;
determine that the fourth sequence number is less than the third sequence number;
modify the fourth sequence number to generate a modified fourth sequence number that is greater than the third sequence number; and
send the second key, the fourth data, and the modified fourth sequence number to the coordination node.
3. The distributed storage system of claim 2 , wherein the coordination node is further configured to store the second key, the fourth data, and the modified fourth sequence number to the transaction log.
4. The distributed storage system of claim 1 , wherein the coordination node is further configured to:
send, to a witness service, a message that the first sequence number is associated with the key; and
store the first data associated with the key to a first persistent storage node of the plurality of persistent storage nodes.
5. A computer-implemented method comprising:
under control of a distributed storage system comprising one or more computing devices configured to execute specific instructions:
storing, in a transaction log, first data associated with a key and a first sequence number;
receiving a write request comprising second data associated with the key;
generating a second sequence number;
obtaining the first sequence number and the first data from the transaction log based at least partly on the key;
determining that the second sequence number is less than the first sequence number;
determining whether the second data is different from the first data; and
performing a storage operation in response to the write request based at least partly on determining whether the second data is different from the first data.
6. The computer-implemented method of claim 5 , wherein performing the storage operation comprises:
modifying the second sequence number to generate a modified second sequence number that is greater than the first sequence number; and
storing, in the transaction log, the second data associated with the key and the modified second sequence number.
7. The computer-implemented method of claim 6 , further comprising:
determining that the write request comprises a particular write request type of a plurality of write request types, and
determining, based at least partly on the write request comprising the particular write request type, that the storage operation is to include modifying the second sequence number.
8. The computer-implemented method of claim 5 , wherein determining whether the second data is different from the first data comprises determining that the first data is equal to the second data, and wherein performing the storage operation comprises acknowledging storage of the second data.
9. The computer-implemented method of claim 8 , further comprising:
determining that the write request is associated with a first write request type of a plurality of write request types, and
determining, based at least partly on the write request being associated with the first write request type, that the storage operation is to include determining whether the first data is equal to the second data.
10. The computer-implemented method of claim 5 , wherein performing the storage operation comprises rejecting storage of the second data.
11. The computer-implemented method of claim 5 , further comprising:
sending, to a witness service, a message that the first sequence number is associated with the key; and
storing the first data associated with the key to a first persistent storage node of the distributed storage system.
12. The computer-implemented method of claim 5 , further comprising:
receiving a second write request comprising third data associated with the key;
generating a third sequence number;
obtaining the first sequence number from the transaction log based at least partly on the key;
determining that the second sequence number is greater than the first sequence number; and
storing the third data and the third sequence number in the transaction log.
13. A system comprising:
computer-readable memory storing executable instructions; and
one or more computing devices programmed by the executable instructions to at least:
store, in a transaction log, first data associated with a key and a first sequence number;
receive a write request comprising second data associated with the key;
generate a second sequence number;
obtain the first sequence number from the transaction log based at least partly on the key;
determine that the second sequence number is less than the first sequence number;
modify the second sequence number to generate a modified second sequence number; and
perform a storage operation in response to the write request based at least partly on the modified second sequence number.
14. The system of claim 13 , wherein to perform the storage operation, the one or more computing devices are programmed by further executable instructions to:
store, in the transaction log, the second data associated with the key and the modified second sequence number, wherein the modified second sequence number is greater than the first sequence number.
15. The system of claim 14 , wherein the one or more computing devices are programmed by further executable instructions to:
determine that the write request comprises a particular write request type of a plurality of write request types, and
determine, based at least partly on the write request comprising the particular write request type, that the storage operation is to include modifying the second sequence number.
16. The system of claim 13 , wherein the one or more computing devices are programmed by further executable instructions to determine that the first data is equal to the second data, wherein performance of the storage operation comprises generating an acknowledgement of storage of the second data without storing the modified second sequence number in the transaction log.
17. The system of claim 16 , wherein the one or more computing devices are programmed by further executable instructions to:
determine that the write request comprises a single-part write request, and
determine, based at least partly on the write request comprising the single-part write request, that the storage operation is to include determining whether the first data is equal to the second data.
18. The system of claim 13 , wherein to perform the storage operation, the one or more computing devices are programmed by further executable instructions to reject storage of the second data.
19. The system of claim 13 , wherein the one or more computing devices are programmed by further executable instructions to:
send, to a witness service, a message that the first sequence number is associated with the key; and
store the first data associated with the key to a first persistent storage node.
20. The system of claim 13 , wherein the one or more computing devices are programmed by further executable instructions to:
receive a second write request comprising third data associated with the key;
generate a third sequence number;
obtain the first sequence number from the transaction log based at least partly on the key;
determine that the second sequence number is greater than the first sequence number; and
store the third data and the third sequence number in the transaction log.