Systems and Methods for Referencing Data on a Storage Medium
A storage layer is configured to store data at respective offsets within storage units of a storage device. Physical addresses of the data may be segmented into a first portion identifying the storage unit in which the data is stored, and a second portion that indicates the offset of the data within the identified storage unit. An index of the data offsets (e.g., second portions of the physical addresses) may be persisted on the storage device. The first portion of the address may be associated with logical addresses of the data in a forward index. The forward index may omit the second portion of the physical addresses, which may reduce the memory overhead of the index and/or allow the forward index to reference larger storage devices. Data of a particular logical address may be accessed using the first portion of the physical address maintained in the forward index, and the second portion of the media address stored on the storage device.
1 . An apparatus, comprising:
a storage client comprising an operating system and a file system; and
a storage layer configured to perform storage operations from the storage client on a storage medium comprising a plurality of storage arrays, the storage layer comprising:
a front-end address interface configured to receive from the storage client a front-end address which refers to an identifier used to reference data and/or storage resources;
a translation module configured to map the front-end address to a storage address comprising:
a media address of a storage unit of the storage medium, and
an offset index specifying a relative location of a data packet within the storage unit; and
a storage controller comprising:
a data layout module for storing data and parity in a horizontal arrangement in the plurality of storage arrays, where the data in a horizontal row can be constructed using the parity in that row,
a write module comprising a parity data generator and an ECC write module configured to create ECC codewords, and
a read module comprising an ECC read module configured to detect and correct errors in the data read from the storage medium.
2 . The apparatus of claim 1 , where the storage layer comprises a groomer module configured to reclaim storage resources of the storage medium.
3 . The apparatus of claim 2 , where the groomer module is configured to operate as an autonomous, background process.
4 . The apparatus of claim 1 , where the storage medium comprises a plurality of solid-state storage devices or drives.
5 . The apparatus of claim 1 , where the offset index is stored within a respective storage unit.
6 . The apparatus of claim 1 , where the storage controller is configured to perform compression and/or decompression of write and read data.
7 . The apparatus of claim 1 , where the storage layer is configured to actively scan the storage medium to identify sections that should be taken out of service.
8 . The apparatus of claim 1 , where the storage layer is configured to reconstruct the data by use of the parity data.