Tiering valid data after a disaster recovery operation
Staging data on a storage element integrating fast durable storage and bulk durable storage, including: receiving, at a storage element integrating fast durable storage and bulk durable storage, a data storage operation from a host computer; storing data corresponding to the data storage operation within fast durable storage in accordance with a first data resiliency technique; and responsive to detecting a condition for transferring data between fast durable storage and bulk durable storage, transferring the data from fast durable storage to bulk durable storage in accordance with a second data resiliency technique.
1 . A method comprising:
storing a plurality of commit identities with one or more portions of data written to a storage system;
determining a subset of the plurality of commit identities based on an associated redundancy level that are allowed commit identities;
based on the determination, responsive to a system recovery, determining, by a processing device of a storage controller,
a first set of valid data portions of the one or more portions of data that correspond to the subset that are allowed commit identities; and
a second set of invalid data portions of the one or more portions of data that do not correspond to the subset that are disallowed commit identities; and
based on the determination, transferring the first set of valid data portions of the data from fast durable storage of the storage system to bulk durable storage of the storage system.
2 . The method of claim 1 , wherein the one or more portions of data are shards of a redundant array of independent drives (RAID) stripe.
3 . The method of claim 1 , wherein the one or more commit identities for the one or more portions of data are written into headers for subsequent shards.
4 . The method of claim 1 , wherein the one or more commit identities are written into memory mapped durable storage.
5 . The method of claim 1 , wherein the one or more commit identities are written into durable registers.
6 . The method of claim 1 , further comprising:
identifying a list of disallowed commit identities by reading a most recently committed list of future commit identities, wherein prior to a portion of data being committed, the portion of data is included within the list of disallowed commit identities, and subsequent to the portion of data being committed, the portion of data is included within a list of allowed commit identities and removed from the list of disallowed commit identities.
7 . The method of claim 1 further comprising identifying a list of allowed commit identities and a list of disallowed commit identities from the one or more commit identities, wherein the list of allowed commit identities represents valid content for the storage system.
8 . An apparatus comprising:
a memory; and
a processor, operatively coupled to the memory, configured to:
store a plurality of commit identities with one or more portions of data written to a storage system;
determine a subset of the plurality of commit identities based on an associated redundancy level that are allowed commit identities;
based on the determination, responsive to a system recovery, determine, by a processing device of a storage controller,
a first set of valid data portions of the one or more portions of data that correspond to the subset that are allowed commit identities; and
a second set of invalid data portions of the one or more portions of data that do not correspond to the subset that are disallowed commit identities; and
based on the determination, transfer the first set of valid data portions of the data from fast durable storage of the storage system to bulk durable storage of the storage system.
9 . The apparatus of claim 8 , wherein the one or more portions of data are shards of a redundant array of independent drives (RAID) stripe.
10 . The apparatus of claim 8 , wherein the one or more commit identities for the one or more portions of data are written into headers for subsequent shards.
11 . The apparatus of claim 8 , wherein the one or more commit identities are written into memory mapped durable storage.
12 . The apparatus of claim 8 , wherein the one or more commit identities are written into durable registers.
13 . The apparatus of claim 8 , wherein the processor is further configured to:
identify a list of disallowed commit identities by reading a most recently committed list of future commit identities, wherein prior to a portion of data being committed, the portion of data is included within a list of disallowed commit identities, and subsequent to the portion of data being committed, the portion of data is included within a list of allowed commit identities and removed from the list of disallowed commit identities.
14 . The apparatus of claim 13 , wherein determining the set of valid portions of the data is dependent upon the list of allowed commit identities and the list of disallowed commit identities.
15 . A non-transitory computer readable storage medium storing instructions which, when executed, cause a processor to:
store a plurality of commit identities with one or more portions of data written to a storage system;
determine a subset of the plurality of commit identities based on an associated redundancy level that are allowed commit identities;
based on the determination, responsive to a system recovery, determine, by a processing device of a storage controller,
a first set of valid data portions of the one or more portions of data that correspond to the subset that are allowed commit identities; and
a second set of invalid data portions of the one or more portions of data that do not correspond to the subset that are disallowed commit identities; and
based on the determination, transfer the first set of valid data portions of the data from fast durable storage of the storage system to bulk durable storage of the storage system.
16 . The non-transitory computer readable storage medium of claim 15 , wherein the one or more portions of data are shards of a redundant array of independent drives (RAID) stripe.
17 . The non-transitory computer readable storage medium of claim 15 , wherein the one or more commit identities for the one or more portions of data are written into headers for subsequent shards.
18 . The non-transitory computer readable storage medium of claim 15 , wherein the one or more commit identities are written into memory mapped durable storage.
19 . The non-transitory computer readable storage medium of claim 15 , wherein prior to a portion of data being committed, the portion of data is included within a list of disallowed commit identities, and subsequent to the portion of data being committed, the portion of data is included within a list of allowed commit identities and removed from the list of disallowed commit identities.