Test data verification for an ordered event stream storage system
Data verification technology for ordered event stream (OES) events written into an ordered event stream storage system is disclosed. The verification technology provides perfect reliability. The verification technology further requires low storage overhead in comparison to typical checksums, storing replicated data, etc. Test event data can be generated in a reproducible manner based upon determined OES metadata. OES metadata can be determined from input received via a user interface, via characteristics of an OES storage system, etc., and can be stored for later use in data verification. The test event data can be stored to a portion of an OES storage system under test. The stored test event data can subsequently be verified by using the stored OES metadata to regenerate test event data for comparison to the stored test event data. The test event ordering can be verified via sequence information included in the stored test event data.
1. A system, comprising:
a processor; and
a memory that stores executable instructions that, when executed by the processor, facilitate performance of operations, comprising:
determining metadata for an ordered event stream storage system, wherein the metadata comprises an ordered event stream identifier, an indicator of ordered event writing parallelism, and a count of test events to be generated;
generating content for storage via test events, wherein the generating is based, at least in part, on the metadata;
writing test events comprising the content to the ordered event stream storage system; and
storing the metadata to facilitate a future verification of the content stored via the test events.
2. The system of claim 1 , wherein the generating the content based on the metadata comprises reproducibly generating the content.
3. The system of claim 2 , wherein passing a portion of the metadata into a quasi-random reproducible sequence of bytes function reproducibly generates the content.
4. The system of claim 1 , wherein the operations further comprise storing the metadata to facilitate a later verification of the content stored via the ordered event stream storage system.
5. The system of claim 4 , wherein the storing the metadata facilitates a future verification of the content stored via the ordered event stream storage system.
6. The system of claim 1 , wherein the metadata further comprises a size of a test event to be generated.
7. The system of claim 1 , wherein the ordered event stream storage system is not modified to facilitate data verification prior to performance of the operations, and wherein the writing the test events is performed using conventional ordered event stream operations.
8. The system of claim 1 , wherein the writing the test events results in writing to an identified portion storage of the ordered event stream storage system, and wherein the identified portion storage is the target of data verification testing.
9. The system of claim 1 , wherein the operations further comprise:
regenerating the content as regenerated content based on the metadata;
receiving the content from the ordered event stream storage system; and
verifying the content based on comparing the content and the regenerated content.
10. The system of claim 9 , wherein the regenerated content is identical to the content previously generated for storage via the ordered event stream storage system.
11. The system of claim 9 , wherein the metadata is retrieved from metadata storage.
12. The system of claim 9 , wherein an order of events is verified based on a portion of the content comprising sequence information.
13. The system of claim 9 , wherein the operations further comprise indicating a verification failure in response to a mismatch being determined to be present between the content and the regenerated content.
14. The system of claim 9 , wherein the operations further comprise initiating a mitigation response in response to a mismatch being determined to be present between the content and the regenerated content.
15. A method, comprising:
generating, by a system comprising a processor, content for storage via test events of an ordered event stream, wherein the generating is based, at least in part, on metadata corresponding to an ordered event stream storage system to be tested, and wherein the content comprises an ordered event stream identifier, an indicator of ordered event writing parallelism, a count of test events to be generated, and sequence information; and
initiating, by the system, writing of the test events comprising the content into the ordered event stream storage system, wherein storage of the test events via the ordered event stream storage system facilitates a future data verification of the content.
16. The method of claim 15 , further comprising:
regenerating, by the system at a future time, the content as regenerated content based on the metadata;
initiating, by the system at the future time, reading the content from the ordered event stream storage system; and
verifying, by the system at the future time, the content based on a result of comparing the content and the regenerated content.
17. The method of claim 15 , wherein the verifying comprises verifying an order of the reading the content from the ordered event stream storage system, and wherein the verifying the order is based on the sequence information comprised in the content read from the ordered event stream storage system.
18. A non-transitory machine-readable medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising:
populating test events for storage via an ordered event stream with content reproducibly determined, at least in part, from metadata corresponding to an ordered event stream storage system to be tested, wherein the content comprises sequence information, and wherein the metadata comprises an ordered event stream identifier, an indicator of ordered event writing parallelism, and a count of test events to be generated; and
causing the ordered event stream storage system to write the test events comprising the content into the ordered event stream storage system, wherein storage of the test events via the ordered event stream storage system facilitates a data verification of the content after the storage of the test events.
19. The non-transitory machine-readable medium of claim 18 , the operations, after the storage of the test events, further comprising:
regenerating the content as regenerated content based on the metadata;
causing the content to be read from the ordered event stream storage system;
verifying an order of the read content based on the sequence information; and
verifying the content based on comparing the content and the regenerated content.
20. The non-transitory machine-readable medium of claim 18 , wherein the operations further comprise storing the metadata to facilitate a later verification of the content stored via the ordered event stream storage system.