Disaster recovery in a streaming data storage system
A streaming data storage system maintains a hierarchical structure of metadata in association with the data chunks of streams, in which the metadata facilitates recovery of the data streams if the streaming data storage system fails. In one implementation, the metadata comprises the pathnames and filenames of the chunks stored in a file storage system or object storage system, in which each pathname represents the epoch and segment of the chunks in the segment, and the chunks' filenames represent the relative ordering of the chunks in the segment. To recover the data stream, the epochs and their segments are recreated, and the segments are repopulated with references to their respective chunks. Once recovered, a new epoch is created with a number of active segments equal to the number of segments of the last recreated epoch, and event appends to the data stream can resume.
1. A system, comprising:
a processor; and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, the operations comprising:
storing a data stream as a hierarchical structure, in which epochs of the data stream are hierarchically higher than segments of the data stream, and the segments are hierarchically higher than data chunks containing event data of the data stream; and
recovering the data stream based on the hierarchical structure, comprising recreating the epochs of the data stream, recreating the segments of the data stream, associating the segments with the epochs based on the hierarchical structure, and associating the data chunks with the segments based on the hierarchical structure.
2. The system of claim 1 , wherein the hierarchical structure corresponds to a file system, wherein the epochs and segments form directories of the file system, and wherein the data chunks are files within the directories.
3. The system of claim 1 , wherein each epoch is identified by a sequential number in the data stream.
4. The system of claim 1 , wherein each segment is identified by a sequential number in the data stream.
5. The system of claim 1 , wherein a segment is associated with two or more epochs.
6. The system of claim 1 , wherein the recreating the segments of the data stream comprises determining whether a segment was created for an epoch, and, in response to determining that the segment was created for the epoch, instructing a segment store to create the segment in association with the epoch.
7. The system of claim 1 , wherein the recreating the segments of the data stream comprises determining that a segment was created for a first epoch, and, in response to determining that the segment was created for the first epoch, adding an identifier of the segment to a second epoch following the first epoch.
8. The system of claim 1 , wherein the operations further comprise sealing the segments, and creating a new epoch with a number of new segments corresponding to a number of segments of a last recreated epoch.
9. The system of claim 8 , wherein the operations further comprise dividing a routing key space evenly among the number of new segments.
10. The system of claim 9 , wherein the operations further comprise receiving an event comprising a routing key and payload, hashing the routing key to determine a matching segment to which the event is mapped, and appending the payload to the matching segment.
11. A system, comprising:
a processor; and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, the operations comprising:
maintaining data chunks that store event data of a data stream, the data stream comprising epochs and segments, in which each data chunk is associated with metadata representing an epoch and a segment corresponding to the chunk and indicating an ordering of the chunk within the segment; and
recovering the data stream, comprising processing metadata of the data chunks to recreate the epochs and segments of the data stream, and to reference respective data chunks within respective segments based on the ordering of the respective data chunks within the respective segments.
12. The system of claim 11 , wherein the metadata associated with a chunk comprises a directory pathname and a filename, the directory pathname representing the epoch and segment of the chunk, and the filename of the chunk indicating the ordering of the chunk within the segment.
13. The system of claim 11 , wherein a segment is identified by a data stream unique value that indicates a relative order of segment creation.
14. The system of claim 11 , wherein the operations further comprise sealing the segments, and creating a new epoch with a number of new segments corresponding to a number of segments of a last recreated epoch.
15. A non-transitory machine-readable medium, comprising executable instructions that, when executed by a processor of a streaming data storage system, facilitate performance of operations, the operations comprising:
storing a first data chunk, comprising first events of a data stream, in a first directory location comprising a first pathname that is based on a first epoch and a first segment associated with the first data chunk;
storing a second data chunk, comprising second events of the data stream, in a second directory location comprising a second pathname that is based on the first epoch and a second segment associated with the second data chunk; and
recovering the data stream based on the first pathname and the second pathname, comprising recreating the first epoch, recreating the first segment in association with the first epoch, recreating the second segment in association with the first epoch, referencing the first data chunk from the first segment, and referencing the second data chunk from the second segment.
16. The non-transitory machine-readable medium of claim 15 , wherein the operations further comprise storing a third data chunk in a third directory location comprising the first pathname, the third data chunk comprising third events of the data stream, wherein the name of the first data chunk within the first pathname and the name of the third data chunk within the first pathname indicate that the first events of the data stream are prior to the third events of the data stream.
17. The non-transitory machine-readable medium of claim 15 , wherein the operations further comprise determining that the recovering of the data stream is complete, and, in response to the determining that the recovering of the data stream is complete, creating a new epoch with a number of active segments corresponding to a number of segments of a last recreated epoch.
18. The non-transitory machine-readable medium of claim 17 , wherein the operations further comprise sealing the segments of the last recreated epoch.
19. The non-transitory machine-readable medium of claim 17 , wherein the operations further comprise dividing a routing key space evenly among the number of active segments.
20. The non-transitory machine-readable medium of claim 19 , wherein the operations further comprise receiving an event comprising a routing key and payload, hashing the routing key to determine a matching active segment to which the event is mapped, and appending the payload to the matching active segment.