Efficient rolling transactions in a data storage system
Efficient rolling transactions for a data storage system is described herein. A method as described herein can include structuring, by a device operatively coupled to a processor, a data stream according to a first structure, the first structure comprising respective stream segments that are associated with respective first key ranges; writing, by the device, a transaction to the data stream, wherein the transaction comprises events arranged in a second structure that is distinct from the first structure, and wherein the second structure comprises respective transaction segments that are associated with respective second key ranges; and merging, by the device, respective ones of the transaction segments into respective ones of the stream segments, resulting in merged transaction segments, in response to the second key ranges of the respective merged transaction segments at least partially matching the first key ranges of the respective ones of the stream segments.
1. A data storage system, comprising:
a memory that stores computer executable components; and
a processor that executes computer executable components stored in the memory, wherein the computer executable components comprise:
a data stream component that maintains a data stream of a first structure, the first structure comprising respective stream segments that are associated with respective first key ranges;
a transaction processing component that commits a transaction to the data stream, wherein the transaction comprises events arranged in a second structure that is distinct from the first structure, the second structure comprising respective transaction segments that are associated with respective second key ranges; and
a merging component that merges respective ones of the transaction segments into respective ones of the stream segments, resulting in merged transaction segments, in response to the second key ranges of the respective merged transaction segments matching at least respective portions of the first key ranges of the respective ones of the stream segments, wherein the merging component merges a first one of the transaction segments into a first one of the stream segments in response to a first one of the second key ranges, for the first one of the transaction segments, matching a first portion of a first key range, for the first one of the stream segments, and wherein the first portion of the first key range comprises less than all of the first key range.
2. The data storage system of claim 1 , wherein the merging component merges the merged transaction segments into the data stream while preserving the first structure of the data stream.
3. The data storage system of claim 1 , wherein the computer executable components further comprise:
a transaction recording component that records the events to the transaction, wherein the transaction recording component generates respective ones of the transaction segments in response to respective ones of the events being assigned to the respective ones of the transaction segments.
4. The data storage system of claim 3 , wherein the transaction recording component assigns the second key ranges to the respective transaction segments such that the second key ranges match respective ones of the first key ranges associated with the data stream at a time at which the respective transaction segments are generated.
5. The data storage system of claim 1 , wherein the merging component further merges a second one of the transaction segments into the first one of the stream segments in response to a second one of the second key ranges for the second one of the transaction segments matching a second portion of the first key range for the first one of the stream segments that comprises less than all of the first key range.
6. The data storage system of claim 5 , wherein the merging component further merges the second one of the transaction segments into the first one of the stream segments further in response to the second one of the transaction segments comprising less than a threshold number of events.
7. The data storage system of claim 1 , wherein the transaction comprises one or more unmerged transaction segments having second key ranges that are distinct from the first key ranges of the respective stream segments, and wherein the merging component alters the first structure of the data stream to the second structure and merges the one or more unmerged transaction segments into the data stream according to the second structure.
8. The data storage system of claim 7 , wherein the merging component returns the data stream to the first structure in response to merging the one or more unmerged transaction segments into the data stream.
9. A method, comprising:
structuring, by a device operatively coupled to a processor, a data stream according to a first structure, the first structure comprising respective stream segments that are associated with respective first key ranges;
writing, by the device, a transaction to the data stream, wherein the transaction comprises events arranged in a second structure that is distinct from the first structure, and wherein the second structure comprises respective transaction segments that are associated with respective second key ranges; and
merging, by the device, respective ones of the transaction segments into respective ones of the stream segments, resulting in merged transaction segments, in response to the second key ranges of the respective merged transaction segments at least partially matching the first key ranges of the respective ones of the stream segments, wherein the merging comprises merging a first one of the transaction segments into a first one of the stream segments in response to a first one of the second key ranges, for the first one of the transaction segments, matching a first portion of a first key range, for the first one of the stream segments, and wherein the first portion of the first key range comprises less than all of the first key range.
10. The method of claim 9 , wherein the merging comprises merging the merged transaction segments into the data stream while preserving the first structure of the data stream.
11. The method of claim 9 , further comprising:
recording, by the device, the events to the transaction; and
generating, by the device, respective ones of the transaction segments in response to respective ones of the events being assigned to the respective ones of the transaction segments.
12. The method of claim 11 , further comprising:
assigning, by the device, the second key ranges to the respective transaction segments such that the second key ranges match respective ones of the first key ranges associated with the data stream at a time of the generating the respective ones of the transaction segments.
13. The method of claim 9 , wherein the merging further comprises merging a second one of the transaction segments into the first one of the stream segments in response to a second one of the second key ranges for the second one of the transaction segments matching a second portion of the first key range for the first one of the stream segments that comprises less than all of the first key range.
14. A non-transitory machine-readable medium comprising computer executable instructions that, when executed by a processor of a data storage system, facilitate performance of operations, the operations comprising:
maintaining a data stream of a first structure, the first structure comprising respective first segments that are associated with respective first key ranges;
committing a transaction to the data stream, wherein the transaction comprises events arranged in a second structure that is distinct from the first structure, and wherein the second structure comprises respective second segments that are associated with respective second key ranges; and
merging respective ones of the second segments of the transaction into respective ones of the first segments of the data stream, resulting in merged second segments, in response to the second key ranges of the respective merged second segments matching at least respective portions of the first key ranges of the respective ones of the first segments, wherein the merging comprises merging a first one of the second segments of the transaction into a first one of the first segments of the data stream in response to a first one of the second key ranges, for the first one of the second segments, matching a first portion of a first key range, for the first one of the first segments, and wherein the first portion of the first key range comprises less than all of the first key range.
15. The non-transitory machine-readable medium of claim 14 , wherein the operations further comprise:
merging the merged second segments into the data stream while preserving the first structure of the data stream.
16. The non-transitory machine-readable medium of claim 14 , wherein the operations further comprise:
recording the events to the transaction; and
generating respective ones of the second segments in response to respective ones of the events being assigned to the respective ones of the second segments.
17. The non-transitory machine-readable medium of claim 14 , wherein the operations further comprise:
merging a second one of the second segments of the transaction into the first one of the first segments of the data stream in response to a second one of the second key ranges for the second one of the second segments matching a second portion of the first key range for the first one of the first segments that comprises less than all of the first key range.
18. The data storage system of claim 1 , wherein a first number of the respective transaction segments is less than a second number of the respective stream segments.
19. The data storage system of claim 18 , wherein the merging component compares the second key ranges of the respective transaction segments to the first key ranges of the respective stream segments, and wherein the merging component merges each of the respective transaction segments into the respective ones of the stream segments in response to determining that each of the second key ranges of the respective transaction segments coincides with corresponding first key ranges of the respective stream segments.
20. The method of claim 9 , wherein a first amount of the respective transaction segments is less than a second amount of the respective stream segments, and wherein the method further comprises:
comparing, by the device, the second key ranges of the respective transaction segments to the first key ranges of the respective stream segments, wherein the merging comprises merging all of the respective transaction segments into the respective ones of the stream segments in response to all of the second key ranges of the respective transaction segments coinciding with respective ones of the first key ranges of the first stream segments.