Recovering data within a unified storage element
Staging data on a storage element integrating fast durable storage and bulk durable storage, including: receiving, at a storage element integrating fast durable storage and bulk durable storage, a data storage operation from a host computer; storing data corresponding to the data storage operation within fast durable storage in accordance with a first data resiliency technique; and responsive to detecting a condition for transferring data between fast durable storage and bulk durable storage, transferring the data from fast durable storage to bulk durable storage in accordance with a second data resiliency technique.
1. A method comprising:
receiving, at a storage device comprising fast durable storage, multiple portions of data from a host computer;
storing, in the fast durable storage of the storage device, one or more commit identities for one or more of the multiple portions of the data that have been successfully written, wherein prior to a portion of data being committed, the portion of data is included in a list of disallowed commit identities, and subsequent to the portion of data being committed, the portion of data is included in a list of allowed commit identities and removed from the list of disallowed commit identities; and
responsive to a system recovery, reading the one or more commit identities from the fast durable storage to determine a set of valid portions of the data.
2. The method of claim 1 , wherein the multiple portions of data are shards of a RAID stripe.
3. The method of claim 1 , wherein the one or more commit identities for the one or more of the multiple portions of data are written into headers for subsequent shards.
4. The method of claim 1 , wherein the one or more commit identities are written into memory mapped durable storage.
5. The method of claim 1 , further comprising:
transferring successfully written portions of the data from fast durable storage to bulk durable storage.
6. The method of claim 1 , wherein determining the set of valid portions of the data is dependent upon the list of allowed commit identities and the list of disallowed commit identities.
7. The method of claim 1 , wherein the one or more commit identities are written into durable registers.
8. An apparatus comprising a computer processor, a computer memory operatively coupled to the computer processor, the computer memory having disposed in it computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of:
receiving, at a storage device comprising fast durable storage, multiple portions of data from a host computer;
storing, in the fast durable storage of the storage device, one or more commit identities for one or more of the multiple portions of the data that have been successfully written, wherein prior to a portion of data being committed, the portion of data is included in a list of disallowed commit identities, and subsequent to the portion of data being committed, the portion of data is included in a list of allowed commit identities and removed from the list of disallowed commit identities; and
responsive to a system recovery, reading the one or more commit identities from the fast durable storage to determine a set of valid portions of the data.
9. The apparatus of claim 8 , wherein the multiple portions of data are shards of a RAID stripe.
10. The apparatus of claim 8 , wherein the one or more commit identities for the one or more of the multiple portions of data are written into headers for subsequent shards.
11. The apparatus of claim 8 , wherein the one or more commit identities are written into memory mapped durable storage.
12. The apparatus of claim 8 , wherein the computer program instructions that, when executed by the computer processor, cause the apparatus to carry out the steps of:
transferring successfully written portions of the data from fast durable storage to bulk durable storage.
13. The apparatus of claim 8 , wherein determining the set of valid portions of the data is dependent upon the list of allowed commit identities and the list of disallowed commit identities.
14. The apparatus of claim 8 , wherein the one or more commit identities are written into durable registers.
15. A computer program product product disposed upon a computer readable medium, the computer program product comprising computer program instructions that, when executed, cause a computer to carry out the steps of:
receiving, at a storage device comprising fast durable storage, multiple portions of data from a host computer;
storing, in the fast durable storage of the storage device, one or more commit identities for one or more of the multiple portions of the data that have been successfully written, wherein prior to a portion of data being committed, the portion of data is included in a list of disallowed commit identities, and subsequent to the portion of data being committed, the portion of data is included in a list of allowed commit identities and removed from the list of disallowed commit identities; and
responsive to a system recovery, reading the one or more commit identities from the fast durable storage to determine a set of valid portions of the data.
16. The computer program product of claim 15 , wherein the multiple portions of data are shards of a RAID stripe.
17. The computer program product of claim 15 , wherein the one or more commit identities for the one or more of the multiple portions of data are written into headers for subsequent shards.
18. The computer program product of claim 15 , wherein the one or more commit identities are written into memory mapped durable storage.
19. The computer program product of claim 15 , wherein determining the set of valid portions of the data is dependent upon the list of allowed commit identities and the list of disallowed commit identities.
20. The computer program product of claim 15 , wherein the one or more commit identities are written into durable registers.