Barrierless snapshots
A method is provided comprising: generating a snapshot of a volume in a storage system; generating a snapshot marker and adding the snapshot marker to a transaction log of the storage system; scanning the transaction log to identify one or more records that have been added to the transaction log before the snapshot marker, and marking each of the identified records as a splitflush record; flushing any record that is stored in the transaction log by: (a) detecting whether the record is marked as a splitflush record, (b) when the record is marked as a splitflush record, copying the record both to a data store and to the snapshot, and (c) when the record is not marked as a splitflush record, copying the record to the data store only.
1. A method comprising:
generating a snapshot of a volume in a storage system;
generating a snapshot marker and adding the snapshot marker to a transaction log of the storage system;
scanning the transaction log to identify one or more records that have been added to the transaction log before the snapshot marker, and marking each of the identified one or more records as a splitflush record; and
flushing any record that is stored in the transaction log by: (a) detecting whether the record is marked as a splitflush record, (b) when the record is marked as a splitflush record, copying the record both to a data store and to the snapshot, and (c) when the record is not marked as a splitflush record, copying the record to the data store only.
2. The method of claim 1 , wherein any record that is stored in the transaction log is assigned a logical sequence number (LSN) and the snapshot marker is also assigned an LSN, and scanning the transaction log includes identifying one or more records in the transaction log that have lower LSNs than the snapshot marker.
3. The method of claim 1 , wherein at least some of the identified one or more records that are marked as splitflush records are flushed in parallel with one another.
4. The method of claim 1 , further comprising:
generating a recovered copy of the transaction log in response to a failure in the storage system;
detecting that the snapshot marker is present in the recovered copy of the transaction log; and
scanning the recovered copy of the transaction log to identify one or more records that have been added to the transaction log before the snapshot marker, and marking each of the one or more records that are identified in the recovered copy of the transaction log as a splitflush record.
5. The method of claim 1 , further comprising unmarking a respective one of the identified one or more records when a new record is added to the transaction log that is associated with a same logical block address as the respective record.
6. The method of claim 1 , wherein the transaction log includes a ring buffer having a tail and a head, and adding the snapshot marker to the transaction log includes adding the snapshot marker at the head of the ring buffer.
7. The method of claim 1 , wherein marking any of the identified records as a splitflush record includes modifying metadata corresponding to the identified record to reference both the snapshot and the volume.
8. A system comprising:
a memory; and
one or more processors operatively coupled to the memory, the one or more processors being configured to perform operations of:
generating a snapshot of a volume in a storage system;
generating a snapshot marker and adding the snapshot marker to a transaction log of the storage system;
scanning the transaction log to identify one or more records that have been added to the transaction log before the snapshot marker, and marking each of the identified records as a splitflush record; and
flushing any record that is stored in the transaction log by: (a) detecting whether the record is marked as a splitflush record, (b) when the record is marked as a splitflush record, copying the record both to a data store and to the snapshot, and (c) when the record is not marked as a splitflush record, copying the record to the data store only.
9. The system of claim 8 , wherein any record that is stored in the transaction log is assigned a logical sequence number (LSN) and the snapshot marker is also assigned an LSN, and scanning the transaction log includes identifying one or more records in the transaction log that have lower LSNs than the snapshot marker.
10. The system of claim 8 , wherein at least some of the identified one or more records that are marked as splitflush records are flushed in parallel with one another.
11. The system of claim 8 , wherein the one or more processors are further configured to perform the operations of:
generating a recovered copy of the transaction log in response to a failure in the storage system;
detecting that the snapshot marker is present in the recovered copy of the transaction log; and
scanning the recovered copy of the transaction log to identify one or more records that have been added to the transaction log before the snapshot marker, and marking each of the one or more records that are identified in the recovered copy of the transaction log as a splitflush record.
12. The system of claim 8 , wherein the one or more processors are further configured to perform the operation of unmarking a respective one of the identified one or more records when a new record is added to the transaction log that is associated with a same logical block address as the respective record.
13. The system of claim 8 , wherein the transaction log includes a ring buffer having a tail and a head, and adding the snapshot marker to the transaction log includes adding the snapshot marker at the head of the ring buffer.
14. The system of claim 8 , wherein marking any of the identified records as a splitflush record includes modifying metadata corresponding to the identified record to reference both the snapshot and the volume.
15. A non-transitory computer-readable medium storing processor-executable instructions, which when executed by at least one processor cause the processor to perform operations of:
generating a snapshot of a volume in a storage system;
generating a snapshot marker and adding the snapshot marker to a transaction log of the storage system;
scanning the transaction log to identify one or more records that have been added to the transaction log before the snapshot marker, and marking each of the identified one or more records as a splitflush record; and
flushing any record that is stored in the transaction log by: (a) detecting whether the record is marked as a splitflush record, (b) when the record is marked as a splitflush record, copying the record both to a data store and to the snapshot, and (c) when the record is not marked as a splitflush record, copying the record to the data store only.
16. The non-transitory computer-readable medium of claim 15 , wherein any record that is stored in the transaction log is assigned a logical sequence number (LSN) and the snapshot marker is also assigned an LSN, and scanning the transaction log includes identifying one or more records in the transaction log that have lower LSNs than the snapshot marker.
17. The non-transitory computer-readable medium of claim 15 , wherein at least some of the identified one or more records that are marked as splitflush records are flushed in parallel with one another.
18. The non-transitory computer-readable medium of claim 15 , wherein the processor-executable instructions, when executed by the at least one processor, further cause the perform the operations of:
generating a recovered copy of the transaction log in response to a failure in the storage system;
detecting that the snapshot marker is present in the recovered copy of the transaction log; and
scanning the recovered copy of the transaction log to identify one or more records that have been added to the transaction log before the snapshot marker, and marking each of the one or more records that are identified in the recovered copy of the transaction log as a splitflush record.
19. The non-transitory computer-readable medium of claim 15 , wherein the processor-executable instructions, when executed by the at least one processor, further cause the processor to perform the operation of unmarking a respective one of the identified one or more records when a new record is added to the transaction log that is associated with a same logical block address as the respective record.
20. The non-transitory computer-readable medium of claim 15 , wherein the transaction log includes a ring buffer having a tail and a head, and adding the snapshot marker to the transaction log includes adding the snapshot marker at the head of the ring buffer.