Recovery from partial device error in data storage system
Techniques are provided to recover from partial device errors of storage devices in a data storage system. A storage control system manages a storage device which comprises storage capacity that is logically partitioned into segments of equal size. The storage control system groups at least some of the segments of the storage device into a segment group. Each segment of the segment group is configured to store one or more data items and associated metadata items. The storage control system generates a parity data segment based on the segments of the segment group, and persistently stores the parity data segment in association with the segment group. In response to detecting a storage device error associated with a corrupted segment of the segment group, the storage control system utilizes the parity data segment and non-corrupted segments of the segment group to recover at least one missing data item of the corrupted segment.
1. A method, comprising:
managing, by a storage control system, a storage array comprising a plurality of individual hardware storage devices, wherein each individual hardware storage device is configured to store data and comprises storage capacity, wherein at least a portion of the storage capacity of at least one individual hardware storage device of the storage array is logically partitioned into segments of equal size;
grouping, by the storage control system, at least some of the segments of the at least one individual hardware storage device of the storage array into a segment group, wherein each segment of the segment group is configured to store one or more data items and associated metadata items;
generating, by the storage control system, a parity data segment based on the segments of the segment group of the at least one individual hardware storage device;
persistently storing, by the storage control system, the parity data segment in association with the segment group; and
in response to detecting, by the storage control system, a partial storage device error resulting from a corrupted segment of the segment group of the at least one individual hardware storage device, utilizing the parity data segment and non-corrupted segments of the segment group of the at least one individual hardware storage device to recover at least one missing data item of the corrupted segment and storing at least one recovered data item on the at least one individual hardware storage device;
wherein generating the parity data segment based on the segments within the segment group comprises extracting metadata items of all segments of the segment group and computing parity over all the extracted metadata items separate from the data items.
2. The method of claim 1 , wherein the segments of the at least one individual hardware storage device comprise log segments of a log-structured array which is managed by the storage control system.
3. The method of claim 1 , wherein persistently storing the parity data segment in association with the segment group comprises storing the parity data segment in a persistent memory device which resides on a storage server node which hosts the storage control system.
4. The method of claim 1 , wherein utilizing the parity data segment and the non-corrupted segments of the segment group to recover at least one missing data item of the corrupted segment comprises:
accessing the parity data segment associated with the segment group;
accessing the metadata items from each non-corrupted segment of the segment group;
reconstructing the metadata items of the corrupted segment using the parity data segment and the metadata items of the non-corrupted segments of the segment group;
utilizing the reconstructed metadata items to identify a missing data item in the corrupted segment;
accessing a replica of the data item, which is identified as the missing data item, from a replica storage device; and
storing the replica of the data item on the at least one individual hardware storage device to thereby rebuild the corrupted segment.
5. The method of claim 4 , wherein accessing the replica of the data item from the replica storage device comprises accessing the replica of the data item from a given replica storage device which is included with the at least one individual hardware storage device in a replication group of a mesh replication system.
6. The method of claim 1 , wherein computing parity over all the extracted metadata items separate from the data items comprises:
grouping the extracted metadata items from each segment of the segment group into respective blocks of metadata items for the respective segments; and
computing the parity data segment for the segment group by computing parity over the blocks of metadata items for the respective segments.
7. The method of claim 1 , wherein metadata items of respective data items comprise information associated with the respective data items, wherein the information comprises one or more of an identity of the respective data items, sizes of the respective data items, offset locations of the respective data items within a given segment, and locations where replica copies of the respective data items reside.
8. An article of manufacture comprising a non-transitory processor-readable storage medium having stored therein program code of one or more software programs, wherein the program code is executable by one or more processors to implement a method which comprises:
managing, by a storage control system, a storage array comprising a plurality of individual hardware storage devices, wherein each individual hardware storage device is configured to store data and comprises storage capacity, wherein at least a portion of the storage capacity of at least one individual hardware storage device of the storage array is logically partitioned into segments of equal size;
grouping, by the storage control system, at least some of the segments of the at least one individual hardware storage device of the storage array into a segment group, wherein each segment of the segment group is configured to store one or more data items and associated metadata items;
generating, by the storage control system, a parity data segment based on the segments of the segment group of the at least one individual hardware storage device;
persistently storing, by the storage control system, the parity data segment in association with the segment group; and
in response to detecting, by the storage control system, a partial storage device error resulting from a corrupted segment of the segment group of the at least one individual hardware storage device, utilizing the parity data segment and non-corrupted segments of the segment group of the at least one individual hardware storage device to recover at least one missing data item of the corrupted segment and storing at least one recovered data item on the at least one individual hardware storage device;
wherein generating the parity data segment based on the segments within the segment group comprises extracting metadata items of all segments of the segment group and computing parity over all the extracted metadata items separate from the data items.
9. The article of manufacture of claim 8 , wherein the segments of the at least one individual hardware storage device comprise log segments of a log-structured array which is managed by the storage control system.
10. The article of manufacture of claim 8 , wherein the program code for persistently storing the parity data segment in association with the segment group comprises program code for storing the parity data segment in a persistent memory device which resides on a storage server node which hosts the storage control system.
11. The article of manufacture of claim 8 , wherein
the program code for utilizing the parity data segment and the non-corrupted segments of the segment group to recover at least one missing data item of the corrupted segment comprises:
program code for accessing the parity data segment associated with the segment group;
program code for accessing the metadata items from each non-corrupted segment of the segment group;
program code for reconstructing the metadata items of the corrupted segment using the parity data segment and the metadata items of the non-corrupted segments of the segment group;
program code for utilizing the reconstructed metadata items to identify a missing data item in the corrupted segment;
program code for accessing a replica of the data item, which is identified as the missing data item, from a replica storage device; and
program code for storing the replica of the data item on the at least one individual hardware storage device to thereby rebuild the corrupted segment.
12. The article of manufacture of claim 11 , wherein the program code for accessing the replica of the data item from the replica storage device comprises program code for accessing the replica of the data item from a given replica storage device which is included with the at least one individual hardware storage device in a replication group of a mesh replication system.
13. The article of manufacture of claim 8 , wherein the program code for computing parity over all the extracted metadata items separate from the data items comprises:
program code for grouping the extracted metadata items from each segment of the segment group into respective blocks of metadata items for the respective segments; and
program code for computing the parity data segment for the segment group by computing parity over the blocks of metadata items for the respective segments.
14. The article of manufacture of claim 8 , wherein metadata items of respective data items comprise information associated with the respective data items, wherein the information comprises one or more of an identity of the respective data items, sizes of the respective data items, offset locations of the respective data items within a given segment, and locations where replica copies of the respective data items reside.
15. A system, comprising:
a storage server node comprising at least one processor, and a system memory configured to store program code, wherein the program code is executable by the at least one processor to implement a storage control system which is hosted by the storage server node and which is configured to:
manage a storage array comprising a plurality of individual hardware storage devices, wherein each individual hardware storage device is configured to store data and comprises storage capacity, wherein at least a portion of the storage capacity of at least one individual hardware storage device of the storage array is logically partitioned into segments of equal size;
group at least some of the segments of the at least one individual hardware storage device of the storage array into a segment group, wherein each segment of the segment group is configured to store one or more data items and associated metadata items;
generate a parity data segment based on the segments of the segment group of the at least one individual hardware storage device;
persistently store the parity data segment in association with the segment group; and
in response to detecting a partial storage device error resulting from a corrupted segment of the segment group of the at least one individual hardware storage device, utilize the parity data segment and non-corrupted segments of the segment group of the at least one individual hardware storage device to recover at least one missing data item of the corrupted segment and storing at least one recovered data item on the at least one individual hardware storage device;
wherein generating the parity data segment based on the segments within the segment group comprises extracting metadata items of all segments of the segment group and computing parity over all the extracted metadata items separate from the data items.
16. The system of claim 15 , wherein in persistently storing the parity data segment in association with the segment group, the storage control system is configured to store the parity data segment in a persistent memory device which resides on the storage server node.
17. The system of claim 15 , wherein
in utilizing the parity data segment and the non-corrupted segments of the segment group to recover at least one missing data item of the corrupted segment, the storage control system is configured to:
access the parity data segment associated with the segment group;
access the metadata items from each non-corrupted segment of the segment group;
reconstruct the metadata items of the corrupted segment using the parity data segment and the metadata items of the non-corrupted segments of the segment group;
utilize the reconstructed metadata items to identify a corrupted data item in the corrupted segment;
access a replica of the data item, which is identified as the missing data item, from a replica storage device; and
store the replica of the data item on the at least one individual hardware storage device to thereby rebuild the corrupted segment.
18. The system of claim 17 , wherein in accessing the replica of the data item from the replica storage device, the storage control system is configured to access the replica of the data item from a given replica storage device which is included with the at least one individual hardware storage device in a replication group of a mesh replication system.
19. The system of claim 15 , wherein the segments of the at least one individual hardware storage device comprise log segments of a log-structured array which is managed by the storage control system.
20. The system of claim 15 , wherein in computing parity over all the extracted metadata items separate from the data items, the storage control system is configured to:
group the extracted metadata items from each segment of the segment group into respective blocks of metadata items for the respective segments; and
compute the parity data segment for the segment group by computing parity over the blocks of metadata items for the respective segments.